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showing most progress in the school effectiveness program. 
Recommendations include calls for ongoing technical assistance to 
teachers and a statewide review of reading instruction. Appendices 
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Pro#aln 1n Connecticut 



I. INTRODUCTION 

Schools can and do make a difference 1n how much and how well all children 
learn. During the past decade, educational research findings have contributed 
greatly to our understandings of more effective school practices to Improve 

learning, teaching and leadership. p 

\ J 
< Over the past several years, a number of school effectiveness activities 

have been Initiated by the Connecticut Stats Department of Education. Based 

upon research findings, a model and process were developed to help schools 

Increase their effectiveness and Instruments were developed to measure the 

status of school effectiveness characteristics.* To date, 35 have volunteered 

to participate it the Connecticut School Effectiveness Program. 

The purpose of this evaluation 1s to present what has occurred, 1n terms 

of school processes and products, 1n order to Improve program practices at /the 

State Department, district and school levels. 



*The following publications by the Connecticut State Department of Education 
contain extensive Information about the school effectiveness research and 
about the Connecticut School Effectiveness Program: 

Instructional^ Effective Schools: A Model and a Process ' (Gauthler. W. t 
1983) 

Research-Based School Improvement Practices (Shoemaker. J. f 1984) 

Handbook for Use of the Connecticut School Effectiveness Interview 
and Questionnaire (Proctor, C. t VUlanova, W. t 1984) 
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This document 1s Intended to: 

o ) provide Information about ^he degree to which the program 1s 
V accomplishing Its stated purposes; 

/ 1 

o Istlmulate thought and discussion about school Improvement practices; 

o Inform the State Department policy making and dedslop making 
/processes; and 

p contribute to the resear&h and evaluation literature base for school 
effectiveness. - s 
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II. .THEORY INTO PRACTICE 
The Connecticut General Assembly, 1n 1978, required that a comprehensive plan 
be developed for elementary and secondary education. At the same time, the 
General Assembly .made clear concerns about the relatively poor achievement 
of low Income children* A program to Improve school effectiveness provided 
not only a timely response to the legislature, but also a progressive chapter 
/ 1n Connecticut's Comprg.nenslve Plan for Elementary and Secondary Education , 

Connecticut's School Effectiveness Plan 1s based upon the pioneering 
research of Wilbur Brookover, Ronald Edmonds and Lawrence lezotte. The plan . 
became operational 1n the summer of 1980 when, as part of a Department 
reorganization, a unit to Implement school effectiveness was Included 1n the 
Bureau of School and Program Development. By the winter of 1980-81, the 
Involvement of staff from the Bureau of Research, Planning and Evaluation were 
added to help design an evaluation for the program. 

Connecticut's definition of an effective school focuses on the achievement ^ 



performance of low-1ncofoe children. Based oa Edmond's definition fTJTSTi a , 

school 1s effective if, and only 1f, the proportion of low-Income children 

■ 

obtaining mastery 1s the same a', the proportion of middle-Income children 

obtaining mastery. Schools 1n which only middle-Income children achieve 

***** 

mastery are categorized as differentially effective and schools where very few 
low or middle-Income children achieve mastery are classified as Ineffective. 

The Connecticut State Department of Education has selected seven 
characteristics that emerge from the literature on school effectiveness that 
appear to be correlated with studefnt achievement. 
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o Safe and Orderly Environment . There 1s an orderly, purposeful 

atmosphere which 1s free from^the threat of physical harm. However, 
the atmosphere 1s not oppressive and 1s conducive to teaching and 
learning. 

o Clear School Mission . There 1s a clgarly articulated mission of the 
school through which the staff shares an understanding of and a 
commitment to{lnstruct1onal goals, priorities, assessment procedures 
and accountability. 

o Instructional leadership . The principal acts as the Instructional 
leader who effectively communicates the mission of the school to the 
staff, parents and students, and who understands and applies the 
characteristics of Instructional effectiveness 1n the management of 
the Instructional program of the school. 

o H1ah Expectations . The school displays a climate of expectation 1n 
which the staff believes and demonstrates that students can attain 
mastery of basic skills and that staff members have the capability to 
help students achieve such mastery. 

r 

o Opportunity to learn and Student Time on Task . Teachers allocate a 
significant amount of classroom time to Instruction 1n basic skills 
areas. For a high percentage of .that allocated time, students are 
engaged 1n planned learning activities. 

o Frequent Monitoring of Student Progress . Feedback on student 
academic progress 1s obtained frequently. Multiple assessment 
methods such as teacher-made tests, samples of student work, mastery 
skills checklists, criterion-referenced tests, and norm-referenced 
tests are used. The results of testing are used to Improve 
Individual student performance and also to Improve the Instructional 
program. 

o Home-Sc-hoo-T-Re-lat-lons- . Parents-understand and s upport the basic 

mission of the school and are made to feel that.. they have an 
Important role 1n achieving this mission. 

The Connecticut School Effectiveness Program attempts to assure the 

acquisition of mastery 1n reading, writing and mathematics for all children by ^ 

helping schools examine and Improve the characteristics listed 1n the model. 

This 1s accomplished through a voluntary process which helps a principal and 

faculty examine certain characteristics associated with student learning, 

develop an action plan and Initiate long-term change aimed at modifying these 

characteristics 1n a school-based setting. Schools are encouraged to resist 

addressing only one characteristic at a time. Meaningful change 1s seen as 
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addressing Instruction, curriculum, organizational dynamics and community 
Involvement 1n an Integrated way. School Effectiveness Initiatives, must be 
strongly supported by central office personnel but the autonomy of the 
principal and faculty members must be protected. The process Includes the 
following steps: 

o Initial Contact . A facilitator from the Bureau of School and Program 
Development 1s designated as the person who will provide ass1stance\ 
to the school. The Initial contact 1s made with the superintendent \ 
of schools and a meeting takes place. * The research background, the \ 
model and the process for working with specific schools 1n the 
district are described. There 1s an emphasis that school 
effectiveness will not resolve personnel problems. 

o Dialogue and Commitment . If the superintendent agrees to proceed, 
the facilitator meets wlth^a designated principal. Principals 
express their willingness to ""participate either to their 
superintendents or to the Department staff directly. The program 1s 
discussed 1n detail and the facilitator stresses the Importance of 
the principal to the eventual success of the process. The 
facilitator Indicates that this 1s an opportunity for the principal 
. and faculty to carry out a. self analysis of their school. *The 
assessment team will help them collect Information, but 
extrapolations from the data must be made by principal and staff. 
The final action must be theirs. At the end of the exchange, the 
principal and faculty decide whetKer or not to participate 1n the 
program. 

o Assessment . * The assessment takes place over a two-to-three day 
period by staff from the Bureau of School and Program Development 
using The Connecticut School Effectlvenss Interview , a 67 Item 
structured Interview that probes the seven school effectiveness 
,> characteristics (Appendix A ), and the Connecticut School 
Effectiveness Questionnaire (Appendix B ) t a 100-Uem, 
paper-and-pendl Instrument that parallels the structured Interview. 
Achievement data are gathered and analyzed by socioeconomic class 
subsets (Appendix C ). Archival data such as mastery skills 
checklists, report cards, and student handbooks are collected and 
Integrated (Appendix D ). These data are compiled and profiles are 
prepared by staff from the Bureau of Research, Planning and 
Evaluation. The results are shared first with the principal and then 
with the entire faculty. 



♦Assessment of several schools 1n the School Effectiveness Project was 
conducted using a New York based assessment because 1n the early stages of 
this project, the Connecticut assessment Instruments had not not yet been 
developed. 
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o Problem - Solving Institute . The principal and school action planning 
team are taken to a site away from the school setting. Over a 
two-to-three-day period, they analyze the data gathered during the 
assessment- The facilitator helps the school personnel analyze data 
from various sources and helps the team develop an action plan based 
upon the assessment findings. 

o Implementation . The principal and the action planning team return to 
the school and present the action plan to the entire faculty and 
Implementation occurs according to the time line delineated In the 
action plan. 

To date, 35 elementary, middle schools and one Junior high have begun the 

process. The 14 schools which have been participating the longest (at least 

two years prior to June 1983) are the focus for this evaluation report. As a 

statewide Initiative, school effectiveness has several distinguishing features' 

which are critical to the development of an evaluation design: 

o Volunteerlsm Implies, not only that schools are not obligated to 

participate, but also that schools are not accountable (at least to 
the Department) for what they accomplish or fall to accomplish* 

o The Department provides no direct funding to participating schools 



and few, 1f any, staff development resources beyond the Initial 
assessment and action plan phases. 

Since school effectiveness essentially 1s 1n the hands of each 
participating school, the pace and Intensity of Improvement 1s 
dictated by the willingness, commitment and readiness of the school 
faculty. 



o Since basic skills achievement Is defined by each school's testing 
program, basic skills achievement across 14 schools 1s measured by a 
variety of different standardized norm-referenced tests . 

Given these constraints, all 14 schools agreed to participate 1n the 

evaluation process. The schools are located 1n seven districts: 



\ 



District 



Number of Schools 



Merlden 

Hartford 

New London 

New Britain 

Vernon 

Coventry 

Grlswold 



4 

2 
2 
2 
2 
1 
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III. EVALUATION OESIGN 
Developing an evaluation design for the Connecticut School Effectiveness 
Program posed some unique challenges. Unlike other educational Innovations, 
school effectiveness has emerged as a home-grown enterprise. At Its core, the 
evaluation of school effectiveness, at least 1n Connecticut, 1s the study of 
teachers and principals putting research Into practice with all the Inevitable 
bumps, twists, and U-turns. If there are 14 schools 1n the evaluation sample, 
there are likely to be 14 individual responses to what the research says, how 
the assessment data are used, and what the goals and objectives should be 1n 
the project. 

It 1s Imperative that the design fit the wide range of Implementation 
efforts. However, 1n this case, schools and the Department were developing 

4 

Implementation strategies at the same time as the Implementation of the 
program was occurring. The original 14 schools had a hands-on learning 
experience. Thus, school effectiveness initiatives are much less systematic 
1n the original 14 schools than 1n schools which entered later. 

The school effectiveness evaluation design 1n Connecticut Includes both 
quantitative and qualitative methods for gathering outcome Information on both 
school processes and products, Including, but not limited to, student 
achievement. The Intent of this evaluation 1s to obtain a better 
understanding of what works and what doesn't work In the program. The 
evaluation 1s formative 1n that Its 1 raj or purpose 1s to Improve, program 
practices. — 

The evaluation design contains four components: 

1. Context Information . Context Information 1s defined as baseline 
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data from the assessment Instruments gathered at the beginning of the 
program. The school's Initial status on the seven characteristics, 
student achievement" scores on standardized tests, and the school s 
action plans are Included. 

Process Information . Process Information describes to what extent 
and by what means school faculties have Implemented their action 
plans. Process data were collected 1n several ways: 

o Case studies lasting four to six days allowing time for 

1n-depth, semi-structured Interviews and school and classroom 
observations were conducted 1n four of the 14 schools. 

o One-day case studies allowing for samples of teachers to be 
Interviewed were conducted 1n the remaining ten of the 14 
schools. A Practice Profile (Appendix E ) which emphasizes the 
degree to which a school's action plans are being accomplished, 
was used 1n the one-day visits. 

o Interviews were conducted with the state facilitators and with 
district administrators. 

o School-based archival and evaluation data were reviewed. 

S' thottl-Based Outcomes .' The assessment of school-based outcomes 1s 
utilized to document changes that have occurred In the school as a 
result of school effectiveness efforts. Oata sources for this review 
Include document reviews, semi-structured Interviews, pretest and 
posttest results of the Connecticut School Effectiveness 
Questionnaire, achievement data over time, and case study Information. 

r.ross-slte Impact . The focus of this evaluation component 1s to 
assess the Impact of school effectiveness across all 14 sites. In 
addition, school effectiveness Impact 1s measured over at least a 
three-year period. 
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The assessment of the Impact of school effectiveness will primarily focus 
on the following evaluation questions: 

t 

o What action plans have been developed to address the school 
effectiveness, characteristics (context evaluation)? 

o To what extent has the school effectiveness program been Implemented 
(process evaluation)? 

o To what extent has the Implementation of school* action plans promoted 
changes In school organizational practices, climate and Instruction 
over time (school-based evaluation)? 

o Are there significant changes over time 1n reading and mathematics 
achievement scores for low-Income and other^students (school -based 
evaluation, cross site evaluation)? 

o Are there significant changes over time 1n the presence of the seven 
school effectiveness characteristics as measured by the Connecticut 
School Effectiveness Questionnaire (school-based evaluation)? 

» 

o What 1s the relationship among the school effectiveness 

characteristics, achievement outcomes, and case study findings 
(cross-site evaluation)? 

Charts 1 and 2, on the next two pages, outline the evaluation design. '• 
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CONNECTICUT SCHOOL EFFECTIVENESS PROJECT EVALUATION COMPONENTS, QUESTIONS AND DATA SOURCES 



Content- 



-Process- 



1*. ..What Is the status of the school 
on the school effectiveness 
characteristics? 

2. What action plans have been 
developed to address the school 
effectiveness characteristics? 

3* Is there a relationship between 
SES status (low and middle Income) 
and student achievement? 

4.* What type of Information exists 
to support the school effective- 
ness characteristics? 



1* Connecticut Interview and 
Questionnaire 

2. School Action Plans 

3. Norm-referenced Standardized Tests 
4* Archival Oata 



1. To what extent has the school 
Implemented It's action plans? 

2* What modifications were made In 
the Implementation of the school's 
action plan? 

3. What constraints exist that may 
have Impeded Implementation? 



DATA SOURCES 

1. Liaison Interview 

2. Evaluation of Action Plan 

3. Archival Data/Self Report 



School -based Outcomes/ 
-Cross-site Outcomes* 

1* To what extent has the 
implementation of school 
action plans promoted changes 
In school organizational 
practices* climate* and 
instruction over time? 

2. Are there significant 
changes over time in reading 
and mathematics achievement 
scores for low income and* 
other students? 

3. Are there significant changes 
over time in the presence of 
the school effectiveness 
character! sties? 

4. What is the relationship 
among the school effectiveness 
characteristics* achievement 
outcomes and case study 
findings? 



la. Four to Six Oay Case Study 

lb. One Day Practice Profile 

2. Norm-referenced Achievement 
# Test 

3. Pre test/post test Questionnaire 

4* All of the above plus, archival 
data , I g 



lit iter adc > U* sine , stli* by*. ,1n1 ^ata ... the *. r ir«g ♦ ac.w~; ah -choc. ,itos.. 



Chart 2 

< 

DATA SOURCES FOR THE EVALUATION COMPONENTS ANO DATA COLLECTION ACTIVITIES 



CONTEXT INFORMATION (YEAR 1) ? 

• Interview 
Questionnaire* 

• Achievement Profile 

- Archival Oata 

- Action Plan 



Time of Assessment 



Beginning of Project 



PROCESS INFORMATION (YEAR 2) 

- Liaison Interview 

- Archival Data 

- Evaluation of Local Action Plan 



Spring of Second Year 



SCHOOL-BASED OUTCOMES (YEARS 2 AND 3) 

- Questionnaire 

- Achievement Profile 

- Archival Data 

- Evaluation of Local Action Plan 

- On-site Review (case study) 
* - Other Supporting Data 

(state and local) 

CROSS S.ITE OUTCOMES (YEAR 3+) 

- Questionnaire 

- Achievement 
Archival 



Pretest/Posttest 
Yearly, Spring 
Continuous 

Spring of Third Year 
Winter/Spring of Third Year 
When Available 



Biennial Administration 
Yearly /Spring 
Continuous 



- On-site Review (4-6 day case study) W1nter/Spr1ng of Third Year 



OVERALL SUMMARY OF DATA COLLECTION ACTIVITIES 



Data Sources 

Achievement, Standardized Tests 
Connecticut Interview 
Connecticut Questionnaire 

One-Day Case Study (Practice Profile) 



Four-to-S1x-Day Case Study 



Archival Data 

Other Supporting Data (state and local) 



Date Collection Schedule 
Yearly 

Initial Assessment 

Pretest (Initial), Posttest 

(end of 2nd year) 
End of Third Year, 
.Intermittent after 

Initial Assessment 
End of Third Year, 

(Intermittently, 

1f resources permit) 
Continuous 
Continuous 
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IV. THE CHARACTERISTICS OF SCHOOL EFFECTIVENESS 
The Connecticut School Effectiveness Questionnaire has been developed to 
assess seven dimensions of effective schools: safe and orderly environment, 
clear school mission. Instructional leadership, high expectations, opportunity 
to learn, frequent monitoring of student progress, and home-school relations. 
A pretest and posttest administration strategy was used to assess changes In" 
the seven characteristics over time. 

Change over time In the seven characteristics of school effectiveness 1s 
based upon principals' and teachers' recorded perceptions on the 
questionnaire. Of the 14 schools participating _1n the evaluation, pretest and 
posttest questionnaire results were available for ten schools. In these ten 
schools the questionnaire was administered for the first time between January 
and June of 1982 and for the second time between June of 1983 and June of 
1984. The differences 1n these dates reflect different times In which schools 
entered the school effectiveness process. Generally, the pretest and posttest 
dates span about one year to a year and a half from one testing time to the 
other. The evaluation question of Interest 1n this analysis Is: 

Are there significant changes over time 1n the presence of the seven 
characteristics as measured by the Connecticut School Effectiveness 
Questionnaire? 

Findings 

The pretest and posttest questionnaire analysis for each Individual school 1s 
presented 1n Tables 1 through 7 1n Appendix F. Table 8, on the next page, 
presents a summary of Tables 1 through 7. In addition. Information about the • 
Intensity of Implementation and the focus of the school's action plans have 
also been Included in the table. 
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Table 8 



Summary of Significant* Changes in the Seven Character! stics b 
Over Time, Intensity of Implementation 3 and Action Plan Focus 



School 


Safety 


Kiss Ion', 


leader- 
ship 


Expec- 
tations 


Tin* 


Moni- 
toring 


Howe 


1 n t »nt \ ♦ v of 

Implementa- 
tion Index* 


A 


Significant 
Change^ 


Significant 
Change* 


significant 
Change* 


Significant 
Change* 




Significant 
Change 


Significant 
Change 


High 


6 




Significant 
Change 


i 


b 


b 


Significant 


Significant 
Change 


federate 


c 


Significant 
Change* 




b 




Significant 
Change 




b 


Moderate 


D 


b % 


Significant 
Change 


b 




Significant 
Change * - 




Significant 
Change 


Moderate 


e 


b 


Significant 
Change 


Significant 
Change 




b 


Significant 
Change 


b 


Nigh 


F 


b 






b 




Significant 
Change 




Moderate 


K 


Significant 
Change 


Significant 
Change . 


Significant 
Change* 




Significant- 
Change 


b 


Significant 
Change* 


High 


L 


b 












Significant 
Change* 


low 


n 




Significant 
Change* 


b 




J 




Significant 
Change 


low 


N 
















low 










AGGREGATE SUMMARY 










Significant 
Change 


Significant 
Change 


Significant 
Change 


NO 

Change 


Significant 
Change 


Significant 
Change 


Significant 
Change 


Moderate 



" Schools G. H f I and J had no posttest questionnaire, thus are not Included above, 
•implementation Index determined by the collective Judgmnts if SCA facilitators and 
bAaion^lan focus determined by archival data review and case stwe!y Interviews. 
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Highlights of the findings 1n change over time on the seven school 
effectiveness characteristics follow: 

o Clear School Mission and Home/School Relations showed significant 
positive change 1n six schools. 

o Monitoring Student Progress showed significant positive change 1n 
four schools. 

o Safe and Orderly Environment, Leadership and Opportunity to 
learn/T1me on Task showed significant positive change* 1n three 
schools. 

o Expectations showed significant positive change In one school. 

o In School A, six of the seven characteristics showed significant 
positive change. In one school, School K, five of the seven 
characteristics showed positive change. 

o In three schools, Schools B, D and E, three of the characteristics 
showed significant positive change. 

o In two schools. Schools C and H, two of the characteristics showed 
significant positive change. 

o In two schools. Schools F and L, one of the characteristics showed 
significant positive change. 

o In one school, School N, none of the characteristics showed 
significant positive change. 

o Overall significant positive change was found 1n 26 of a possible 70 
cells 1n the table. 

o In the aggregate analysis, six of the characteristics changed 

significantly and one, Expectations, did not change significantly 
from Time 1 to Time 2." The overall analysis shows that across all 
schools there was a significant Improvement 1n the perceptions of the 
staff on six of the seven school effectiveness characteristics. 

The Intensity of Implementation Index ranked schools as low, moderate or 

high and was determined by the collective judgments of Department facilitators 

who worked most closely with the schools and the evaluators who conducted case 

studies and reviewed archival data. Of the three schools. A, E, and K, which 

ranked high on the Intensity Index, two schools, A and K, also showed the 
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highest number of positive changes on the questionnaire. Of the three 
schools, U H and N, which ranked low on the Intensity Index, two schools, L 
and N, showed the fewest positive changes on the questionnaire- School L 
changed on only one characteristic, while school N had no significant changes. 

Action Plan focus was recorded 1n 24 cells 1n Table 8. School A had the 
most action plans (4), the most positive changes on the questionnaire (6) and 
ranked high 1n Intensity of Implementation. 

Of the 24 action plans recorded, nine (38%) corresponded to significant 
changes on the questionnaire. That 1s, schools that developed actions plans 
related to specific characteristics showed a significant change 1n those same 
characteristics 1n 38% of the cases. In addition, half of the ten schools 
have action plans documented 1n the areas of Safe and Orderly Environment and 
1n Leadership. Less than half of the schools recorded action plans 1n the 
remaining five characteristics. 

Questionnaire Summary 

There 1s consistency between the Intensity of Implementation and measured 
significant positive change. The Implications are that the school 
effectiveness characteristics may be more alterable when teachers and 
principals directly focus attention upon them. 

There 1s less consistency between the recorded action plan focus and 
measured significant positive change. 

Except for the extremes (schools where changes were most numerous and 
least numerous), recorded action plans only moderately (38%) related to 
significant changes on the questionnaire. At least twtf possible explanations 



can be offered. In most schools, the action planning process was present at 
the beginning but did not continue 1n a formal, documented way throughout the 
life of the program. Even though there are no recorded action plans 1n many 
schools, there was a continuing focus on the characteristics. Secondly, the 
characteristics should not be considered discrete even though the 
questionnaire forces them Into a discrete mold. A focus on one characteristic 
is very likely to cause a change 1n another characteristic. For example, 
where principals stress the Importance of coordination and articulation of 
curriculum across and between grades and where principals emphasize the 
Importance of test scores, changes on the Clear School Mission characteristic 
may result, even though the focus originated from the leadership of the 
principal rather than from an action plan addressing Clear School Mission. 
At the same time, measured change may occur on the Clear School Mission 
characteristic even though the curriculum changes are the result of a 
principal 's Instructional leadership. 

The case study data (4-6 day case study) provides another perspective of 
the Impact of the school effectiveness characteristics on schools. In these 
analyses the researchers attempted to track: What happens 1n shools as 
1mplementa1on begins? How do teachers and principals translate research 
finding and the school effectiveness characteristics Into day-to-day 
activities? and What are the perceptions of school faculties about the Impact 
of these changes? The following section will highlight the case study 
findings focusing on each characteristic separately. 
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Translating the Seven Characteristics 

What happens 1n schools as Implementation begins? How do teachers and 
principals translate the research findings and the school effectiveness 
characteristics Into day-to-day activities? 

1. Safe and Orderly Environment 

The safe and orderly environment characteristic was both a "safe* and a 
popular place for schools to try out school Improvement. Most faculties 
addressed this characteristic 1n their beginning action plans and their 
activities reflect the wide variety of schools 1n the project. For example, 
1n one urban school, security guards were hired, locks and buzzers were 
Installed on doors, and closer associations were formed with the police. In 
one suburban school, rewards were established for good«cH1zensh1p, the school 
got painted and bulletin boards were made more attractive and prominent. In 
almost all schools, discipline policies were reassessed and expanded to 
Include rules for the playground , cafeteria , auditorium and other 
nonlnstructlonal spaces. The "everybody .shares 1n the responsibility" Idea 
was reaffirmed as all faculty were 

2. Clear School Mission 

The clear school mission characteristic was addressed Jn about half of the 
schools but, for the most part, there was a lack of Intense focus 1n this area 
which refers to academic focus. Grade-level skills 1n reading and mathematics 
were developed In one. school, there was attention to curriculum articulation 
between and among grades 1n a second, and a single textbook series 1n reading 
was Instituted In a third. 

In most schools, curriculum activities such as developing goals and 
objectives and developing textbook selection practices were underway prior to 
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participation 1n schoo'l effectiveness. If there was a lack of attention to 
this characteristic, 1t was 1n aspect of extenders curriculum activities to 
determine grade level skills which all students would be expected to master 
and to help teachers Improve their Instructional skills. 
3. Instructional Leadership 

Principals are highly visible 1n thls^ project. Whereas teachers are able, 
to remain anonymous 1n the assessment process and work collectively 1n the 
Implementation phase, there 1s only one principal 1n each school and the » 
principal's leadership qualities often become the focus of attention. 

Principals Instituted many changes. They (V) Increased the quantity and 
quality of formal teacher observations; (2) devoted faculty meetings to 
curriculum and Instruction topics; (3) sought out and arranged for' more 
frequent staff development and they became 'more open" and ■more supportive", 
and (4) paid more attention to time on task. In almost all schools, the 
principal minimized or eliminated* Intensions by the public address system 
or notes to teachers during Instructional time. They showed an Increased 
respect for the teaching process by these actions. Principals became more 
adept 'with "problem solving techniques, Increased their personal contact with 
staff and became more visible throughout the school. 

Teachers note the changes. They describe their principals as "truly 
committed, caring, zealous, sensitive, considerate and mellow,"; people who 
also "displayed clear, strong leadership," "held teachers accountable," and* 
■required lesson plans." 

The combination of "shared decision making" and "strong principal 
Instructional leadership" Influenced the growth of colleglalUy. By 
colleglalUy 1s meant a partnership among school staff, Including the 

principal. 1n the entire process of change-fron the Initial Identification of 

- 
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Issues to Implementation and modification of activities. Principals can "give 



up" a strategy of autonomous decision making, and simultaneously enhance their 

leadership skills. Here are two examples: 

"I've noticed a real change 1n how the principal talks about the 
observation and how he writes things up. " He doesn't mention the bad 
things; he Isn't being critical 1n your Informal observations the way he 
used to be. For the formal district evaluation, he says what he'll be 
looking for. The purpose 1s to get us to think about why we're doing what 
we do. And I think that 1f we're doing something right, then we ought to 
know why we're doing 1t right, so that we can keep on doing It*. 

and. • • 

The principal was making an effort to lessen her control a bit. "We had 
meetings with the administrator and told heriiow we felt, and what we were 
unable to do because of what" she had done," reported a staff member ♦ "And 
she In turn discussed how we had treated her, and that door opened up so 
that we could see her point of view and she could look at our point of 
view and this way there was more understanding. She tried, and we tried, 
and 1t seemed to work.* 

*It was Interesting to see things happen, to watch her change, to become 
more involved with the staff, to become more a part of tjie staff and not 
Just a separate entity,* commented a planning team member » "At the end of 
that school year we were beginning to try* to work things through • The 
office doors were open a bit more. Teachers seemed to be allowed to do 
. more." * 

4. High Expectations 

Many teachers and principals addressed their expectations' about what, how 

much and how well students can achieve as a function of race or economic 

background. One of the most popular staff development sessions was the 

Teacher Expectations and Student Achievement (TESA) Program/ In three 

schools, as many as 853C of the teachers received training' 1n TESA over a two 

» 

year period. In other schools, teachers were merely Introduced to ' le 
concept, or only a few teachers underwent extensive training. 

For example, 1n one school assessment, the data Indicated that teachers 
lacked high expectations for all of the children 1n their classes* After 
considering this Issue, they agreed to Investigate the research on 





expectations and then Involve themselves 1n an Intensive series of workshops 
to help alter their attitudes and behaviors. The facilitator helped to 
arrange for TESA, an In-service program that continued for several months and 
was funded by the district. 

Although the district was unable to provide substitute coverage for 
teachers to observe one another, an Integral part of training, the principal, ' 
eager to have her teachers pursue this opportunity, chose to^cover-the classes 
herself . The teachers were most appreciative. 

After the formal training was completed, participants reported tt\at they 

continued to pay attention to Issues of teacher expectations, and began to 

talk to other staff members about what they had learned. 

■We became Involved 1n reading articles to become more open-minded 1n 
thinking and sharing with the staff the results of our findings- Not 
preaching to them, Just sharing and trying to change attitudes, as to how 
they saw children In the classroom* Not to see then as children who had 
failed first grade, as a failure, but feeling that a new year had begun 
and the youngster had the ability 1f we would Just try and tap 1t." 

Only five teachers and the principal frc^f this school participated, but 

the training had a powerful Impact on them, and they spread the word to their 

colleagues. 

Nonpartld pants who heard about the training and Its message from their 

principal and colleagues were equally aware of and sensitive to this Issue. 

■In the past, 1f a child couldn't do something, we kind of patted him on 
the head and said, 'He'll try later*. By the time later came, perhaps 
we'd forgotten about 1t. And If you don't keep after 1t, It's Just lost 
and they don't get U. Before they (the experts) used to say, 'Don't get 
the child upset. Just leave h1nr alone, If he can't answer, you Just let 
him sit-there. Later on he'll do It.' But that wasn't the case. And 
now, 1f a child Just sits there, we keep on coming back until he finally 
gets 1t. And children show a responsibility now for giving an answer.* 

S. Opportunity to Learn/T1me on Task - 

Making the most of allocated time became a formal activity 1n most 

participating schools. Judicial use of the public address system, adherence 
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to time schedules for classes to begin and end, and attention to reducing time 
for nonlnstructlonal tasks occurred overnight 1n most participating schools. 

In several schools, staff did a serious examination of the coordination of 
Instruction for students who receive extra attention from bilingual teachers 
or remedial teachers. 

u 

In one school, 50% of the staff participated 1n "t1me-on-task" workshops ^ 
after school. As a follow-up to the workshops, teachers were scheduled to 
observe each other while teaching and to code students' time on task '1n an 
organized fashion. Teachers discussed the results with and learned from each 
other. 

6. Frequent Monitoring of Student Progress 

In almost all participating schools, teachers and principals began to pay 
more attention to test results. In two districts, central office 
administrators designed procedures to dlsaggredate scores by "low Income" and 
"other" for the Individual schools. 

One school Implemented a new mathematics program that Included careful 

monitoring of students progress. According to the principal: 

Almost all teachers are now using the same structured approach. The book 
comes with structured monitoring sheets that allow teacher to keep careful 
track of how kids are doing by skUV areas. They now can see 1f a whole 
group 1s getting It or a subgroup might be pulled together for special 
Instruction. The monitoring has also drawn teachers to pay special 
attention to low achievers. They used to accept a 60 or 70 on a test as a 
grade for low-achieving youngsters. Now the emphasis 1s on mastery on 
raised expectations for lower-achieving students. 

In another school, the principal and the reading consultant monitor 

student progress closely. They examine and sign off each end-of-level skills 

test for each student. The pr1nc1paVd1splayed a test, a skills record, and a 
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sign-off sheet (which 1s sent home to parent Indicating mastery of skills at 
that level) which he happened to have on his desk for one group of second 
grade students. 

In addition, the reading consultant said they examine the standardized 
tests given yearly to 'corroborate what they know about kids* adding that she 
"keeps track of every kid." If students are not making the progress she 
thinks they should, she said she takes 1t_to the principal and often takes 
those students for extra Instruction. She said that teachers have the option 
for "an ongoing diagnosis," to "flag" students for additional help at any 
time. The principal said he examines the standardized tests and compares his 
school performance with others 1n the system. He showed charts which 
demonstrated dramatic Improvement on skills tests 1n areas 1n which students 
had received Instruction. 
7* Parent Involvement 

Parent Involvement received the least amount of specific attention In, the 
development of a school's action plan. It appeared that parent Involvement 
was part of an ongoing activity In schools and thus was not. singled out 1n the 
School Effectiveness Program. 

In two schools, parents were Invited to become part of the process from 
the beginning assessment phase* Special assessment questions were developed 
for parents* Parents 1 results were tabulated and parents were Invited to 
participate on action planning committees. 

In another school, parents logged an Impressive' 1,000 hours of volunteer work 
and helped teachers 1n the classroom by working with students on skills 
reinforcement. 
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V. CASE STUDY FINDINGS: CROSS SITE ANALYSES 

In order to produce detailed descriptions of each program and Its Impact 
around the proposed 1ssues t 10 one-day and three six-day ccse studies were 
completed using qualitative research methods such as document review, 
classroom observations, and 1n-depth semi-structured Interviews. 

The focus of the one-day case^tudles was on the Implementation of each 
school's action plans. A Practice Profile enabled us to measure the 
components of the Implementation of school effectiveness strategies at each 
school site (see Appendix E) . The practice profile technique provided us with 
a way to structure our visits and measure Implementation components of school 
effectiveness strategies at each school' site. 

The focus of the four-to-six day case studies was on the full range of 
process and product outcomes. The purpose was the development of detailed 
descriptions of the three schools studied and the Identification of promising 
practices or key variables that contributed to the overall success of the 
program. Outside researchers trained 1n qualitative methods were responsible 
for carrying out this component of the evaluation. 

The important/central evaluation question was: 

To what extent has the school effectiveness program 
promoted changes In school organizational practices, 
climate and Instruction over time? * 

Analysis of the data suggest that we answer that question best by 

highlighting eight elements 1n the project: the change process, the assessment 

prbcess. Improving 1ntra-school communication, voluntary participation, the 

state facilitator role, student achievement, the district office role and 

program dropouts. (The *-6 day case studies have been provided 1n the 

addendum to this report.) 




The Change Process 

Principals and teachers rarely are entrusted with designing school Improvement 
activities for themselves and so the Connecticut process 1s unique 1n 
providing school staff with that opportunity within the framework of the 
effective schools' characteristics. Case studies reveal that along with the 
opportunity, welcomed, by school staffs, comes a certain degree^of ambiguity. 
Project, goals and process were explained to teachers and principals before 
they voted to engage 1n the School Effectiveness project; yet most 
participants said they were unclear about both at the outset. Instead, staff 
talked about the project as providing a framework for what they were doing 
already, and as a way of sharing responsibility for the school with each other 
and with the administration. Data collected at the school site provided the 
. first occasion for concrete activity 1n the change process. 
The Assessment Process 

The assessment process. Implementation of the questionnaire and Interviews and 
feedback sessions, had a high degree of legitimacy. The Information was 
gathered from the* teachers and principals themselves. Sometimes the data 
confirmed what most teachers expected; occasionally 1t surprised them. Most 
found the data enlightening, even when surprising, although a few claimed that 
they had misunderstood the questions. A teacher 1n one school felt the 
assessment process was Important, "because many 1n the faculty (before the 
assessment) didn't think there was anything wrong . . . People were surprised 
at what came out. Some disagreed with 1t and some blamed 1t on the ambiguity 
of the questions. ." But more often teachers claimed that they agreed with the 
compiled data, and were pleased to find that they were not alone In their • 
concerns or 1n their desire for change. ""The faculty were Intrigued by 1t 
(the data). Prior to that, they had been afraid to say what the problems were 
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and now here they were, being openly discussed* And 1t was good for people to 
see how many felt the same way." 

Teachers In another school had a similar reaction. They had bean seeking 
a way 1n which to change their relationships with their principal. "We were 
at opposite poles with our principal," said one long-time teacher 1n the 
school. "We felt threatened by each other and yet we wanted io work 
together." "We needed tools to work with to settle things, but we didn't have 
them/ said another. Oata t anonymously collected and aggregated by the state, 
gave them a safe opportunity to express their honest opinions about the 
school, their principal and their expectations. "Everybody Hked-the 
Interviews and the forms," said another teacher from this school. "We felt 
good about finally expressing ourselves, and the Interview was so formal that 
you knew they would keep 1t confidential." 

In the data collection phase of the project, the Department too, was 
gaining legitimacy and earning the trust of the school staffs. Legitimacy 
accrued more specifically to th(e facilitator -who worked with the school. On 
those few occasions when' the state was late 1n returning the data to the 
schools, often It was the facilitator or evaluator^ as representative of the 
state, who Incurred the wrath of school staff. 

In these schools and 1n others, prior to the program, teachers and 
principal, both committed to Improving educational opportunities for children 
and Improving their own dally lives 1n school, had found no safe and effective 
way 1n which to address their mutual Issues. Data on paper provided each with 
an opportunity to pursue the situation Individually and then assemble to 
discuss and plan Improvement Interventions. ' 
Improving Communicatio ns 

In virtually every school, teachers and principals discovered from the 
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data (If they did not already know) that communications were less successful 
than they desired. However, prior to the program, they had neither the time 
nor the mechanism with which to address changes 1n this area. The Institute*, 
with the data and the presence of a state facilitator, provided the time, 
space and technical skill to safely address the communication. Issue. 

In two schools. Improved communications became an end In Itself rather 
than a means to accomplish other priorities. As a result, faculties became 
adept In expressing their opinions and feelings with each other but they did 
not choose to communicate about the characteristics of effective schools. 

Communications problems can, and do, cut across many other aspects of 
school life that are related to effectiveness. Poor communications can lead 
to a lack of Instructional leadership, or a fallurejto coordinate Instruction 
In a way that provides maximal learning time for students. In this respect, 
cormiunl cat Ions can be an overreaching Issue that indirectly affects other 
characteristics. As such, It Is worth the attention of the school staffs and 
the project provided this opportunity, 
Voluntary Participation 

The voluntary nature of participation In the School Effectiveness Project Is 
both a help and a hindrance. At the outset, It Insures at a minimum that the 
state has not mandated participation. However, some schools felt a mandate to 
participate from the district. District .administrators In a few towns and 
- cities prodded and- cajoled and pushed schools In need of assistance Into the 
program In the hopes that It would Improve any one of a number of local 
problems. In one school, for example, the hope was that participation 

♦Institute refers to the problem-solving session with faculty representatives 
to develop action plans. 
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would reduce the tension (and the number of grievances) 1n the school and give 
teachers a sense of Involvement 1n decision making. In other schools, 
principals pressed for participation because School Effectiveness "fit 1n" 
with their personal plans which Included, but went beyond schooT Improvement. 
Nonetheless, participation 1n most schools could reasonably be called 
voluntary. 

Voluntary participation helped the process because schools had some level 
of choice; It Interfered with attention to the goals of equity and 
achievement. Schools were reasonably free to shape the program as they chose 
and were not held accountable for achieving those goals. Voluntary 
participation was a potential hindrance because teachers and principals could 
half-heartedly vote to cooperate and then do very little. 
The State Facilitator 

The role of the state facilitator was crucial during the assessment and 
planning phases. Facilitators had to deal with the difficult task of forging 
a team out of a principal and four or five teachers, who had little opportunity 
to work before as a group. At times, due to tensions 1n the schools, staff 
had previously avoided talking to each other. In one school, for example, the 
facilitator was critical In building trust among the team members so that they 
could begin to work on school Issues. In all of the schools, the facilitator 
was critical 1n helping the team to develop their own action plan for reentry 
Into the school, presentation of the data and priorities to the school staff, 
and for helping the team begin to Implement the project. 

For the most part, Connecticut State Department of Education Involvement 
was greatest through this planning/Institute phase. School teams left the 
Institute often with a set of action plans for goals and objectives that they 
would address during the first year. Depending on the composition of the 
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action planning team ar*d the extent of Implementation back 1n th« school, 
these characteristics either did or d1d s not become part of the entire staff's 
vocabulary. In one school, where. the program was confined largely to those who 
were on the planning team, the language of school effectiveness did not spread 
throughout the school. In other schools, participation was broadened, the 
characteristics were discussed at regular faculty meetings, and their presence 
was more apparent 1n the school. \ 



There 1s unanimous agreement that the state facilitators contributed 
greatly to the Initial .success of the assessment and action planning phases. 
However, as the program moved out of the Initial phases, the facilitator's 
role became less clear, considerably less pronounced. There was ambiguity 
about what, 1f anything, was expected of the facilitator. A few schools had 
several different facilitators, each with a unique style, which extended the 
ambiguities. 

It was difficult for teachers 1n several schools to describe the 
facilitators role. Faculty might Indicate that the facilitators "come around 
alot", butthey could not describe -whatT-1n-fac-t-,-fac-1-Htator-$-do — In-a-few 
other schools, teachers could not recall the name of the facilitators, nor 
could they- recall having any contact with that person beyond the action 
planning phase. Some principals stated they "get all the help they want." 



Others stated that "they haven't received any help." 

After the assessment and Institute we have the Impression that 
facilitators "keep the peace" and do not "make waves." Facilitators say they 
provide resource 'materials, attempt, .to broker staff development services, 
serve as a sounding board for Issues and concerns, provide continuing help 

« 

with action planning, and arrange data collection for evaluation. They 
describe an active facilitation role. With the exception of the'two districts 



-28- 



36 



described above, school people would describe 1t as a passive, hands-off role 
that leaves principals and teachers to assume the primary responsibility for 
solving problems* 

In two communities the state facilitator made a significant contribution 
through continued and frequent contacts with ine schools and, when possible, 
brokering of staff development resources- Both communities are served by the 
same facilitator and In both communities; the School Effectiveness Program 1s 
beginning to be Institutionalized at the central office level- In no other 
communities has this occurred. It seems apparent that, when follow-up 
facilitation 1s tried, 1t can work. 
The Achievement Dimension 

One of the roost glaring discrepancies between the way 1n which school staff 
has Interpreted school effectiveness, and the way In which the state 
developers have envisioned Its use, occurs 1n the treatment of student 
achievement. 

The lack of direct focus on achieveme n t 1s partly a r esult of local agenda 
and also a result of the way 1n which the program was described by state 
facilitators to school staff. According to school people, the project was 
billed as a school Improvement project 1n which school staffs would, have the 
opportunity to work on Issues that concerned them. For the staffs 1n most 
participating schools, those Issues were communications and safety. However, 
to be fair. Improved achievement Is the ultimate goal of all that goes on 1n 
schools. Student learning 1s the reason that teachers and principals are 1n 
school each day. 

Furthermore, some facilitators decided against bringing back disaggregated 
achievement data to the staff during the Institute phase and, as a result, a 
few principals and teachers never saw the achievement profiles prepared for 
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their schools. The lack of attention to achievement on the part of some 
facilitators coupled with a choice by some principals and teachers to focus on 
other salient Issues shifted attention away from achievement. Sometimes the 
achievement focus was deferred to gain, entry Into the schools. Other times 
out of a belief that other Issues had to be addressed at a school first 1n 
order to create an atmosphere conducive to a more Intensive Instructional 
focus. Whatever the reasons, the result 1s that school faculty overwhelmingly 
view the school effectiveness project as primarily a self renewal project 
(school Improvement) and not an outcome based project. 
The District Office Role 

The Importance of the school as a social system 1s so dominant in the school 
effectiveness research that the role of the district office either was Ignored 
or was afforded superficial treatment. In Connecticut, up front attention 1s 
paid to gaining the approval of the superintendent and there 1s cursory 

mention of a district office coordinator but once Implementation began, 1n 

* 

four of the seven districts with participating schools, there has been Uttle 
or no attention paid to the district office. 

District Involvement ranges from high to benign neglect. In the most 
Involved case, a district" administrator meets frequently and regularly with 
participating principals. The state facilitator holds schools responsible for 
action plans, provides resources where possible, and aggressively promotes 
school effectiveness. In another case, the superintendent along with the 
board and each district administrator Identified school effectiveness support 
responsibilities as part of their early planning. 

There 1s some evidence to suggest that the degree of central office 
Involvement may make the difference between program survival ; and program 
demise where principal turnover 1s at Issue. School effectiveness began In 
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the fall of 1980 with five principals. Only one of the principals Is 1n that 
same position today. In the four remaining schools, two of the new principals 
continued In the program and two did not. The two continuing schools are In 
districts where central office support and Involvement 1$ high. The two 
schools which dropped out are In districts where central office had little or 
no Involvement and paid little attention to the program when replacing the 
principals. 
Program Dropouts 

Not all schools survive the school effectiveness process. Six of the original 

20 schools have withdrawn from the program leaving 14 for this evaluation. 

There Is some evidence to suggest that the principal Is. key to keeping schools 

In the project and to getting them out. 

o Two schools dropped out because the principal left. 

o Thr^e schools dropped out because serious disagreements arose between 
the principals and the staff. These problems could not be resolved 
and the program was discontinued. 

o One-school-dropped-out-because-the-prlnclpaVdeclded-wlthout 

consultation with staff to terminate the project. 

Clearly, this process Is not for all principals. It appears that 

principals whose leadership behavior may be challenged the most seem least 

willing to adjust to the School Effectiveness process. An equally Important 

Issue yet to be discussed Is the responsibility of the State Department of 

Education to the school and to the district when the process falls. 

Case Study Summary 

School effectiveness has a strong presence 1n many schools, and less of a 
presence 1n others. The program tends to shift 1n and out of prominence In a 
school because 1t depends upon the stance of the principal, the Involvement of 
teachers, the Importance of competing projects, the support of the district 
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administration, and the skill, resourcefulness and time commitment of the 
state facilitator. Throughout the participating schools, there has been 
little suggestion that the program Is a bad Idea or on the wrong track. Where 
1t has been tried, It 1s more likely that school people will complain that the 
t program lacks the resources to fulfill Its promises. Whe.re it has been 
successful, It 1s usually because someone or some group within- the school 
district was strong enough to make It happen; a district administrator/who 
volunteered a school and then shepherded It through a principal or the state 
facilitator who helped guide the school along the path of Improvement. Where 
the program had been least successful, 1t was usually because no one took the 
time or had the skills to help 1t develop. Neither school site personnel,' 
district personnel, nor the state facilitator devoted the time to make 1t work. 



40 

-32- 



ERIC 



VI. ACHIEVEMENT FINDINGS / 

OeMn1t1on of an Effective School 

A school in which the proportion of low Income students performing below 
minimum acceptable levels of basic skill proficiency 1s not greater than 
the proportion of other children 1n the school who perform below such 
levels; and all children 1n the school are'performlng at acceptable levels 
of basic skill proficiency as determined by the application of a generally 
accepted school-based standard* 

The previous section described 1n some detail the variety of approaches 

that were Initiated by schools to become more effective. In taking this 

approach, schools from the outset, developed long term strategies for 

Improvement with the goal of significantly Improving achievement through these 

strategies. However, there are no quick fixes and schools recognized that to 

"really" Impact on student achievement will take time* Therefore, while 

significantly Improved student achievement was the project goal t schools 

generally chose to address organizational arrangements (seven characteristics) 

^1n order to establish conditions which would Impact on student achievements 

Unlike many competency based programs, school effectiveness attempted to treat 

the "causes" rather than the symptoms of low achievement. In this context 

monitoring achievement gains was critical, however; each schools program must 

be evaluated on the basis of both the conditions (strategies) which were put 

1n motion for school Improvement as well as assessing achievement over time. 

The achievement analyses 1n this section of the report provides a 

three-year picture of achievement 1n the areas of reading and mathematics 

across 14 schools. The goal of these- analyses was to evaluate whether a 

positive trend 1n .achievement 1s beginning to occur 1n schools. The overall 

evaluation ^uesttar* t-or these analyses 1s: 

Are there significant overall changes in achievement 1n 
reading and mathematics for low-1rcome and other'students? 

* 

The achievement results 1n this section are organized Into three parts. 

<> 
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First, an aggregate analyses 1n reading and mathematics across all 14 schools 
will be presented. Secondly, the relationship between achievement and school 
effectiveness Implementation and leadership will be explored. Finally, 
achievement will be assessed on a school-by-school basis to Investigate the 
relationship between school effectiveness activities and achievement. 
Aggregating Achievement Data 

Norm-referenced standardized test data were gathered 1n the basljc skills areas 
of reading and mathematics across all grade levels 1n which each school * 
tests. It 1s the policy of the school effectiveness program not to Impose any 
additional testing mandates on the schools, therefore only existing 
standardized test data were used 1n the evaluation. Using existing testing 
sources produces considerable variation 1n the type of test data collected' 
such as: 

o A variety of tests are used . Tesis may Include Iowa Test of Basic 
Skills, Metropolitan Achievement Test, Stanford Achievement Test, 
California Achievement Test, and the California Test of Basic Skills. 

o A.varlety of Wade levels are tested . Some schools test at every 
gfade level, other at alternate grade levels, and somr schools "test 
only at a single grade. 

• * / " « - 

o Te sts are administered at different times during the school year . 
This affects the comparability of th$ normative data from test to 
test - some schools test 1n the fall each year, others 1n the spring 
each year and some test 1n the fall and spring of .each year.t 

All 'these variations 1n the testing data underscore the difficulties 

Inherent 1n state level evaluation which "is based on different tests and 

different testing patterns. Nevertheless, at this time, norm-referenced 

standardized test data 1s the only uniform source of achievement data 

available and must be used. It should be noted that the upcoming state 

mandated mastery test program 1n grades 4, 6 and 8 should serve to overcome 

many of the testing problems because student performance will be measured 

statewide on the same set of basic skills objectives and Items. ~- 
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Data Collection 

Connecticut 1 * definition of school effectiveness and Its process for bringing 
back data to school districts for action planning has directly affected the 
method of data collection used in this evaluation. Test scores 3 re not 
collected 1n this project; rather, the number of students scoring within, each 
decile of the test 1s collected. Deciles divide the test score scale Into ten 
equal percentile units , e.g., 1st percentile to 10th percentile, 11th 
percentile to 20th percentile, . . . 91st percentile to 99th percentile (see 
Appendix C). Therefore, 1n each school the following achievement data have 
been collected over time: 

1. norm-referenced standardV .d achievement test data In reading and 
mathematics at the grade level(s)' 1n which schools test; 

» * 

2. socioeconomic status .of students classified Into two groups, 
low-Income and all "other" children; 

3. the number of students (frequency) scoring within each decile of the 
test, 

Establishing Standards 

For comparison purposes, three levels of achievement have, been created to . 

S ;> 

evaluate student performance: a proficiency standard and a below-grade-level 
and an above-grade-level cutoff. The proficiency standard Identifies the . 1 

minimum acceptable level of basic skill proficiency on a standardized test arid 

f 

was set at the 30th percentile. Impact of the Connecticut School j 

* ' i 

Effectiveness program 1s evaluated by examining whether the proportion of 
ldtf^lncome students scoring below the 30th percentile, has significantly 
lessened over time and whether the discrepancy between low-Income and other 
students has narrowed over time.*. 



♦Socioeconomic classification of students were obtained from school records 
and principal Judgements. 
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To evaluate whether all children (low-Income and other), are performing at 
acceptable basic skill levels, their achievement performance was examined by 
establishing a cut-off at the 50th percentile. This cut-off was used to 
assess change in student achievement for low Income and other students below 
and above grade level. The diagram below Illustrates the three levels of 
achievement used In this evaluation. 

Below Grade Level Above Grade Level 

Proficiency 

1 1 l 1 percentile scale 

IXIle 30X1le 50X1le 99%11e 

These achievement levels have been set somewhat- arbitrarily for this 

evaluation. However, these judgements reflect the Department's attempt to 

operationally define Connecticut's School Effect1veness>def1nit1on. The 

proficiency standard, In particular, has been used as a rule of thumb 1n this 

project for several years. 

Achievement Results 

To examine achievement performance, statewide test data have been aggregated 
across 12 of the 14. participating schools. -Two schools have not continued 
with the program and were not Included In the analyses. The method for 
aggregating test score date across schools Is described 1n 'Appendix G. To 
evaluate overall statewide performance, test data w,ere aggregated for the last 
three years (I.e., SY 1981-1982, SY 1982-1983, and 

sy 1983-1984). Aggregate achievement results across 12 schools are presented 
in Graphs 1 and 2 on Page 40. Figure. 1 on Pages 38 and 39 has been developed 
to guide the reader in understanding the types of analyses presented. 




Figure 1 



INTERPRETING AGGREGATE ACHIEVEMENT OATA 



Findings 



To read the graphic results the following description 1s necessary. 



1. Title - In each content area three separate graphs are Illustrated. 
That 1s the percent of low Income and other Income children are 
charted and compared with respect to examining the percent of students 
falling belo the proficiency standard (30th percentile), percent of 
students scoring below grade level (50th percentile) and the percent 
of students scoring above grade level (50th percentile and above). 
Each graph 1s titled at the top of the page. 

2- Percent of Students - This 1s the percent of students whose score on a 
nationally normed standardized test fell within each of the three 
percentile bands - l-30th percentile, l-50th percentile, 51-99th 
percentl le. 

.3. Data Cycles - At least three years of achievement data have been 

collected 1n all schools. Data cycle one through three represent the 
longitudinal sequence 1n which the data has btp collected. 

The Interpretation of the graphs also require some explanation. Keep 1n 

mind that the direction of the graphic plot as well as the shape of the graph 

(shaded portion) form the basis for Interpretation. For example: 





3 



1 



2 

cycles 



3 



1 . Percent of, students below the 
• Proficiency Standard - The Ideal 
pattern If for the percent of low 
Income students to get progressively 
smaller over time (Data Cycles 1-3). 
For other students, the Ideal pattern 
1s less predictable, however, 1n 
general they should also reduce the 
percent of students scoring below the 
cutoff score over time. The Ideal 
pattern 1s for the low Income 
achievement line to convene with the 
achievement of other student! over 
time. Thus, the -proportion of low 
Income and other students scoring below 
the proficiency standard would 
essentially be the same. At the left 
1s a picture of what an Ideal graph 
might look like. 



Converging lines 
pointing downward 
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Graphic lines some- 
what parallel but 
pointing downwards. 



2. Percent of students~below grade level - 
The' Ideal pattern 1s for the percent of 
low Income students to decrease over 
time. The profile for other students 
should decrease as well. It 1s 
hypothesized that the proportion of low 
Income and other students $cor1ng'below 
this cutoff should decrease at 
' approximately the same rate. * The 
narrowing of the gap between low Income 
and other students should be 
Interpreted as further evidence of 
program Impact. At the left 1s a 
picture of what the graph might look 
like. 
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Percent of students scoring above the 
50th percentile - The Ideal pattern 
would be for the percent of low Income 
students to progressively Increase over 
time. The same pattern should also 
occur with other students. However, 
research would suggest v that the 
Increase 1n percentage would probably 
occur at a greater rate with the other 
Income students. Therefore, a very 
encouraging pattern here would be that 
low Income students and other students 
would Increase their above grade level 
percentages at approximately the same 
rate over time (parallel Improvement). 
At the left 1s a picture of what the 
Ideal graph might look like. 



Graphic lines 
pointing up. 
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Graph 1 

READING RESULTS: Aggregate Summary Over Three Years 
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MATHEMATICS RESULTS: Aggregate Summary Over three Years 
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Achievement Results 1n Reading 

Inspection of Graph 1 reveals that the: 

o proportion of low-Income students scoring below the proficiency 

standard (30th percentile) decreased over time 41 percent (cycle 1) 
to 33 percent (cycle 3). 

o discrepancy (gap) between low T 1ncome and other students around the 
proficiency standard has narrowed* 

o proportion of low-Income students scoring below grade level has 

slightly Increased over time. The percent of other students scoring 
below grade level remained essentially the same. . 

o proportion of low-Income- students scoring above grade level has 

slightly decreased over time. The percent of other students scoring 
above grade level stayed about the same. 

Achievement Results 1n Mathematics' 

Inspection of Graph 2 presents a somewhat different picture of achievement 
and reveals that the: 

o proportion of low-Income students scoring below the 30th percentile 
has declined overtime (36 percent to 24 percent), 

o gap between low-Income and other students around the proficiency 
standard has substantially narrowed. 

o proportion of low-Income students scoring below grade level has 

generally decreased* The gap between low Income students and other 
students lialso narrowing. The pattern of . these scores 1s 
encouraging and suggests that the positive mathematics results extend 
beyond the proficiency standard. 

o proportion of low Income students scoring above grade level has 
generally Increased and the gap. between the other students has 
significantly narrowed. 

See Appendix H for a stat1st1calanalys1s of these findings. 
Summary 

The achievement of low-Income students has Improved 1n reading and 
mathematics. .However, .the, reading pattern Is considerably less encouraging; 
while the percent of low-Income students scoring below the 30X1 le has been 
reduced, the overall reading pattern In comparison with other children around 
the grade-level cutoff shows little growth. To further close the gap, much 
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more work 1s apparently needed 1n this area. The mathematics results show a 
more positive picture. These results Illustrate the goal of school 
effectiveness — significant reduction of students below a mastery standard 
and a narrowing of the gap between low-Income and other students. 
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B. Relationship Between' Achievement and Implementa tion and Between 
Achievement and Leadership 

This section. examines two evaluation dimensions that are central to 
achieving a better understanding of the Impact of school effectiveness. First 
the Impact of Implementation on achievement will be presented. Second, the 
Impact of leadership on achievement will be Investigated. Th*se dimensions 
are not mutually exclusive and are related; however, for purposes of these 
analyses they are treated separately. More specifically, two evaluation 
questions will be addressed: 

1. What Is the effect of Implementation of the school effectiveness 
program on the achievement of low-Income students? 

2. - What 1s th* effect of leadership on the achievement of low-Income 

students? 

The same method for aggregating achievement data 1n section two of this report 
w111.be employed here (see Appendix H). 

Implementation of school effectiveness 1s a very difficult concept to 
measure. As described 1n the previous section, there 1s considerable 
variation 1nUhe way schools Implement school effectiveness programs. 

To examine the Implementation question, the qualitative case study data 
was used to classify schools Into two categories (a) moderate Implementation, 
and (b) h1gn "Implementation. In addition, a practice profile was developed to 
assess the Implementation of each school's- action plan.* These data were also 
used 1n classifying schools with regard to Implementation (see Appendix E). 

♦The practice profile was adopted from the work of Sue Loucks and 1s an out- 
growth of work done on the Innovative Configuration developed at the 
University of Texas. 
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Six schools, on the basis of these data, were classified as having high 
Implementation, five schools were classified as having moderate Implementation 
and three schools did not continue with the program and were not Included 1n 
the analyses. 

Leadership was determined by examining the school faculty's perception of 
their principals leadership on Connecticut's School Effectiveness 
Questionnaire- Leadership on this scile primarily measures principal behavior 
with respect to Instructional Iss.ues. The Instructional leadership mean scale 
scores were rank ordered; the top five schools designated as schools that have 
high leadership and the remaining seven schools were designated as schools 
with average leadership. It should be pointed out that the schools designated 
as having average leadership are not lacking leader- 
ship. It 1s possible that 1f other leadership Instruments were used that the 
rankings of principals may differ. 
Findings 

Graphs 3 through 6 present the achievement analyses based on the 
Implementation Index. Graphs 7 through 10 present the achievement based on 
the leadership rankings. 
. Implementation Highlights 
Reading 

r 

o The reading performance for moderate Implementers of school 
effectiveness show some decline In the proportion of low-Income 
students scoring below the proficiency standard (about 10X) but the 
gap between low-Income and other students remains wide. 

o The reading performance for high Implementers of school effectiveness 
shows little" change 1n reading performance for both low- Income and 
other students, although 1n these schools the percent of students 
scoring below the proficiency standard Is considerably lower than the 
moderate Implementation schools. 
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READING: The Relationship Between Moderate Implementation and Achievemer' 
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MATHEMATICS: . • 

' The Relationship Between Moderate Implementation and Achievement 
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READING: The Relationship Between High Implementation and Achievement 
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MATHEMATICS: The Relationship Between High Implementation and Achievement 
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READING: The Relationship Between Average Leadership and Achievement 
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MATHEMATICS: The Relationship Between Average Leadership and Achievement 
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READING: The Relationship Between High Leadership and Achievement 
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MATHEMATICS: The Relationship Between High Leadership and Achievement 
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Mathematics 

o The mathematics performance for moderate Implemented of school 
effectiveness 1s very similar r to the overall mathematics findings. 
The percent of low-achieving students scoring below the proficiency 
standard has been considerably reduced. Inspection of the below and 
above grade level comparisons are encouraging; I.e., a .reduction 1n 
the proportion of low-Income students scoring below grade level and an 
Increase 1n the proportion of students scoring above grade level. 

o The mathematics performance for high implemented of school 

effectiveness 1s also positive - a lowering of the proportion of low 
Income students below proficiency and an Increase of low-Income 
students scoring above grade level. 

Leadership Highlights 

Reading 

o The reading performance of average leadership schools showed little 
change over the three years. The proportion of low-Income students 
scoring below the proficiency standard remained about the same, the 
gap between low-Income and other students was essentially the same and 
the proportion of low-Income students scoring below and above grade 
level changed slightly. 

o The reading performance of high leadership schools generally 

paralleled the aggregate findings (Graph 1). That 1s, some reduction 
(approximately 9%) 1n the proportion of low-Income students scoring 
below the proficiency standard and some narrowing of the gap between 
low-Income and other students although the gap remains wide. 

Mathematics 

o The mathematics performance of average leadership schools shows little 
change 1n the proportion of low-Income students scoring below the 
proficiency standard. However, the proportion of low-income students 
■scoring below grade* level has been reduced and those students scoring 
above grade level has been increased which is an encouraging trend. 

o The mathematics performance of high leadership schools 1s more 

definitive. The proportion of low-income students scoring below the 
proficiency standards has decreased and the gap between low-income and 
other students 1s closing. The below and above grade level comparison 
seems to reinforce this positive trend. 

See Appendix H for a statistical analysis of the Implementation and 
leadership findings. 
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Summary 

The schools which have been rated as high Implementation schools and the 
schools rated as moderate Implementation schools have somewhat contradictory 
patterns. In reading, the schools rated as moderate show a" greater decrease 
over time 1n the percent of students scoring below the proficiency standard 
than the high Implementation schools. In mathematics, both sets of data ~~ 
patterns between moderate and high Implemented are moving 1n the desired 
d1 recti ons. 

The analyses of achievement on the basis of the leadership category was 
more conclusive. In both reading and mathematics, the high leadership schools 
showed the most progress. The proportion of low Income students scoring below 
the 30th percentile 1n both reading and mathematics was reduced 1n the high 
leadership schools. 

The perception of a school faculty with respect to a principal's 
Instructional leadership appears to be related to student achievement. This 
finding 1s quite consistent with the school effectiveness research which has 
singled out the principal as a critical factor 1n promoting achievement 1n the 
bask skills. 

These results seem to suggest that Implementation has less Influence on 
achievement than does leadership. On closer examination, these results are 
less definitive. Two schools rated as high leadership schools by their 
faculty were rated as moderate Implementation schools. In both schools the 
most amount of progress 1n reading and mathematics was achieved. School 
effectiveness 1n both schools operated as a framework for school reform but 
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1n both schools other Instructional Initiatives were launched, 1n conjunction 
with school effectiveness to address basic skill problems 1n these schools. 

In summary, the complexity of the schools' organization makes the 
Isolation of program factors and the determination of relationships a complex 
undertaking. Apparently, to obtain a better assessment of the relationship 
between Implementation and achievement, the Influence of other program 
Initiatives by the school or the school district must be taken Into account. 
C. School -bv-School Analyses * 
Assessing how all the school effectiveness pieces fit 1s analogous to putting 
together a jigsaw puzzle with some missing pieces. Examining the school 
effectiveness data provides a picture of achievement and a description of the 
activities schools engaged 1n with respect to the school effectiveness 
effort. What Is missing and not Incorporated 1n this study 1s the Influence 
of school district Initiatives regarding basic skill Improvement and the 
Impact of the state focus on basic skills over the last five years. These 
factors and others could affect the overall achievement of schools. To 
.complete the achievement picture an assessment of all relevant Information 
, about the school Including an assessment of Instructional effectiveness 1s 
needed.' Therefore, the goal of the achievement analyses 1s to Identify and „ 
examine relationships that might exist between school effectiveness practices 
and student achievement. 

One evaluation question generally summarizes the school-by-school analysis: 

What 1s the relationship among school effectiveness characteristics, 
achievement outcomes, and case study findings? * 
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To better examine this question, Individual school profiles have been 
developed. The same method of analysis used for the aggregate analysis has 
been used to profile Individual schools. In addition to the achievement 
profiles 1n reading and mathematics, a listing of the schools' action plans, 
performance on the Connecticut School Effectiveness Questionnaire, and a 
description of school effectiveness activities by characteristic has also been 
Illustrated. The portrayal of the Individual school results are Included 1n 
the addendum to this report for review. Examining all these data 1n context 
provides a picture of each school's achievement 1n relationship to Its school 
effectiveness strategies. 

Table 9 attempts to p;ll together Individual school findings of the 14 
schools with respect to the level of Implementation of the school 
effectiveness program, the leadership rating of the^prlndpal , the change In 
the seven school effectiveness characteristics over time and the reading and 
mathematics performance of students in each school. 
Highlights 

Inspection of Table 9 suggests that: 

o High Implementation and high leadership schools (A, E, K) showed the 
most change on the school effectiveness characteristics (three or more 
characteristics were significant). 

o Student achievement 1n high implementation and high leadership schools 
(A, E, K) was positive. In reading, the achievement was rated: good 
(1 school), to excellent (2 schools). In mathematics, the achievement 
was rated excellent (3 schools). . 

> 

o Student achievement 1n two schools (B, F) with moderate Implementation 
and high leadership was also good. One school (B) was rated good 1n 
reading and both schools (B, F) were rated excellent 1n mathematics. 
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Table 9 



The Relationship Between Student Achievement, School Effectiveness- 
Characteristics, Implementation and Principal Leadership 



School* 


Imple- 
ment.. 

xmuca - 


Significant 
Character- 

1 5 X 1C 5 W 


Leader- 
ship 
Rat lnga 


Reading 
Treno c 


Math 
Trend 6 


A 

• 


High 

- 


Safe 

Mice Inn 

n i ) ) i on 

Leader 
High 

Expectation 
Monitor 

nuiiic 


High 

- 


Excellent 


Excellent 


B 


Moderate 


Mission 
Monitoring 

UAma 

no nic 


High 


Good 


Excellent 


C 


Moderate 


Opportunity 
to Learn 

Qaf* AnH 
oq i c a iiu 

Orderly 


Average 


Fair 

- 


Good 


D 


Moderate 


Learn 
Mission 

Home 


Average 


Fair 


Poor 


E 

- 


High 


Mission 

Lcauci 

Monitoring 


High 


Good 


Excellent 


F* 


Moderate 


Monitoring 


High 


Excellent 


Excellent 


& 


High 


-Ho pre/post 


Average 


Fair 


Excellent 


H 


High 


No pre/post 


Average 


Good 


Excellent 


I 


High 


No pre/post 


Average 


Fair 


Poor 


3 


Moderate 


No ore/nost 


Averaae 


Fair 

rill 


Lxce i i en x 


K* 


High 


Mission 

Leader 

Home 


High 

• 


Excellent 


Excellent 


L 


Low 


Home 


Average 


• Excellent 


Poor 


M 


Low 


Mission 


NA 
Home 


Poor 


Good 


■ N" 


Low 




NA 


NA 


NA . 




KEY 

a school - letters correspond with Individual profiles 1n Appendix A. 

b 1mplementat1on Index - derived from qualitative study. 

Significant characteristics - based on the pretest/posttest analyses of 
Connecticut Questionnaire. 

d leadersh1p rating - based on Connecticut Questionnaire. 

e read1ng/math trends - based on a qualitative assessment of -the achievement 
analyses over time. ' ( 

excellent - reduction In the percent of low Income students scoring 
below the proficiency standard and a significant reduction 1n 
the discrepancy between low Income and other students. 

- - good - reduction 1n the percent of low Income students scoring below 
the proficiency standard but np consistent reduction 1h the 
discrepancy between low Income and other children over time. 

fair - Uttle to no change 1n the percent of low Income students 
scoring below the proficiency standard and almost no change 1n 
the gap between these groups. 

poor - an Increase 1n the proportion of students scoring below the 
30th percentile 1s evident and the gap between low Income and 
other students has either stayed the same or widened . 

♦School F has only low Income students. 
♦School K has only "other" Income students. 
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o Three schools (G, H, I) rated as having high Implementation and 
average leadership and had somewhat mixed achievement results. 
Reading achievement was rated from fair (2 schools) to good (1 
school).. Mathematics had a somewhat more positive pattern; 1 school 
was rated poor and 2 schools were rated excellent. 

o Schools rated as having moderate Implementation and average leadership 
(C, 0, J) were rated fair 1n reading and had. mixed results 1n 
mathematics. In mathematics, one school was rated poor, one school 
was rated good and one school was rated excellent. 

o Schools rated as having low Implementation had generally fair to poor 
results 1n reading and poor to good results 1n mathematics. 

o Generally, all districts reported the mean achievement 1n the schools 
1s rising 1n reading and mathematics over time. * 

Summary 

The data presented 1n Table 9 Illustrates some of the frustration and 
complexity with attempting to link achievement data to our understandings of 
what a school may or may not have accomplished 1n school effectiveness. 
School performance 1s more than the sum total of school effectiveness 
activities 1n a school. For example, a school may have had low Implementation 
with regard to school effectiveness but high Implementation on other projects 
(such as a special project 1n reading and/or mathematics, e.g., School F) that 
might account for achievement gains. Additionally, attention to bask skill 
achievement has been a state focus over the last five years through the EERA 
program. EERA data 1s qulte^conslstent with our overall findings that 
considerable growth has occurred 1n the area of mathematics. Moreover, staff 
development 1n the area of mathematics has been a high priority In the state 
over time. 

What 1s encouraging about these data 1s that schools which have progressed 
the most 1n the school effectiveness program appear to exhibit a greater 
degree of achievement for low Income students than less Involved schools. It 
seems reasonable to hypothesize that school effectiveness program In some 
schools has had an Influence on school achievement. 
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VII. OVERALL SUMMARY ANO CONCLUSIONS 



The following highlights summarize what we have learned as a result of this 
evaluation: 

o Changes are more likely to occur 1n areas such as, clear school 
mission, home/school partnerships, monitoring student progress, 
school safety and leadership. 

o Changes seem less likely to occur 1n areas addressing particular 
teaching techniques and other classroom practices. The Implication 
1s that Instructional changes may require more focussed and Intensive 
staff development and may take longer to accomplish. 

o A single school can Influence and set Into motion d1str1ctw1de 
policies and procedures for d1str1ct-w1de school effectiveness 
efforts. In five districts, school effectiveness adoption 1n one 
school led to school effectiveness adoption 1n several schools with 
accompanying district and board commitment. 

o The Connecticut assessment process and action planning Institutes 
establish clear and valuable boundaries for the direction of school 
effectiveness efforts Without exception, the assessment and action 
planning phases solidified the commitment from those who were already 
supportive and helped to enlist the more active Involvement and 
support from those teachers who were equivocal about the school 
effectiveness effort. 

o More effective communication as a staff goal 1s obviously crOdal to 

the development of colleglallty. However, when communication becomes 

the goal rather than the means to accomplish the goal, the emphasis 

of the school Improvement efforts are compromised. . 

o Volunteerlsm 1s a central theme ln Connecticut^ project. 

Volunteerlsm has Its advantages and disadvantages. On the plus side, 
volunteerlsm: 

- Kelps build colleglallty. In almost all our schools, teachers 
have gained a voice 1n the decisionmaking of the school. 

- Helps to establish program continuity and renewal. 
On the minus side: 

- Some of the most needy schools which have some of the most needy 
students do not volunteer. 
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- Accountability for program outcomes 1s more difficult to 
establish. 

o In mathematics the p-oportlon of low-Income students scoring below 
the 30th percentile has decreased from 36 percent to 24 percent. 

' • Also, the discrepancy between the proportion- of low-Income students • 
achieving minimum proficiency (30th percentile) and the proportion of 
all other students achieving minimum proficiency has narrowed from 19 
percentage points to 10 percentage points. 

o In reading, the proportion of low-Income students scoring below the 
30th percentile went from 41 percent to 32 percent. The discrepancy 
between the proportion of low-Income students achieving minimum 
proficiency (30th percentile) and the proportion of all other 
students achieving minimum proficiency has narrowed from 25 
percentage points to 17 percentage points. 

o What 1s encouraging- about the achievement data 1s that schools that 
demonstrate the greatest achievement gains also were the schools 
which showed the most progress 1n the school effectiveness program. 
Less Involved schools generally did much more poorly. 

The findings are a story of contrasts. There 1s support for the Ideas 
that teachers and principals can make focused school changes and these changes 
will be accompanied by higher student achievement for low Income children. 

There 1s harmony 1n some schools where everything Improves — achievement, 
the presence of the characteristics, and the Internal capacity for self 
renewal among the staff. But there 1s dissonance 1n other schools, — 
achievement may go up for one group and down for another. The characteristics 
which Improve may not be the ones which were given the most attention. 

There 1s certainty about some factors — the* -Importance of the principal's 
behavior and commitment and the Importance of the beginning assessment and 
action planning phases. But there 1s ambiguity about other factors — the 
role of the state facilitator, the availability of resources for staff 
development, the nature and extent of district office support and the effect 
of volunteerlsm. 
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School effectiveness has a strong presence 1n many schools, and less of a 

presence 1n others. Whether^ effective or not In addressing the goals of the 

» 

school staff and the state developers, the program tends to shift In and out ' 
of prominence In a school because It depends upon the stance of the principal, 
the Involvement of teachers, the Importance of competing projects, the support 
of the district administration, and the skill, resourcefulness and time 
commitment of the state facilitator. 

Throughout the participating schools, there has been little suggestion 
that the program 1s a bad Idea or on the wrong track. Where It has been 
tried, 1t 1s more likely that school faculty will complain that the program 
lacks the resources to fulfill Its promises. Where It has been successful, 1t 
1s usually because someone or some group within the school district had been 
strong enough to make 1t happen; a district administrator who volunteered a 
school and then shepherded 1t through; a principal; the state facilitator who 
helped guide the school along the path of Improvement. Where the program had 
been least' successful, 1t was usually because no one took the time or had the 
skills to help 1t develop. Neither school site personnel, district personnel, 
nor the state facilitator devoted the time to make It work. 

Schools can change and School Effectiveness provides the opportunity for 
change. Schools 'can Improve 1f efforts are purposeful, practices are guided 
by research findings, and there 1s support with district and state resources. 
But change takes time, hard work and commitment. 
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VIII. POLICY IMPLICATIONS ANO RECOMMENDATIONS 

The Connecticut School Effectiveness Program has been an experiment - an 
experiment 1n new roles for Department staff and 1n new collaboratlves within 
the Department and between and among districts and schools.. The program never 
enjoyed (nor was burdened by) the Inherent accountability factors common to 
most Department Initiatives - no school was obliged to participate, no monies 
were-awarded to participating schools and no preconceived outcomes were 
required. The School Effectiveness Process 1s still evolving. Evaluation 
Information can and should be used for policy formulation and decision making; 
1t 1s hoped that the Implications of the findings and the following 
recommendations will Inspire and stimulate dialogue about the future of this 
unique program. 

Reconwendatlon 1: Criteria for Pa rticipation 

School Effectiveness was Intended primarily for schools which serve large 
numbers of low-Income children. However, the most needy schools do not always 
volunteer. Moreover, little attention has been paid to the low Income 
criterion as virtually all schools who asked to participate were selected. 

It 1s recommended that mere stringent criteria for participation be 
developed and adhered to 1n order to focus resources 1n the most needy schools, 
Reconwiendatlon 2: Facilitator Role 
' The facilitator role 1s clearly defined and practiced at the dialogue and 
Initiation stage, at the needs assessment phase and through the Institute and 
action-planning phase. The role becomes less clearly defined and ranges from 
extensive Intervention to hands off as teachers and principals move toward 
Implementing their action plans and evaluating their progress. 
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It is recommended that the facilitator role be fully developed and 
extended throughout the process and that training be provided for consultants 
who assume this role. 

» 

Recommendation 3: Instructional Focus 

In schools where the characteristics are present to a high degree, there 1s 
present also the readiness and confidence among the staff to become more 
effective teachers. However, 1n most cases, extensive and elaborate technical 
assistance 1s needed - assistance far beyond the facilitator role. 
Furthermore, In most participating schools there has been a lack of attention 
to Instruction as a specific goal of the School Effectiveness process. 

It 1s recommended that ongoing technical assistance be made available to 
help teachers Improve their Instructional effectiveness* 
Recommendation 4: Validation/Recognition 

Several schools have been participating In the program for as long as three 
years. At present, there are no procedures for reward and recognition for 
schools which have attained effectiveness status. 

It 1s recommended that a two-phase process of vafldat1cnr/recogn1t1on be 
established: 

A. Phase I - Validation/recognition criteria would emphasize the process 
of renewal and reform with attention to: 

a. Evidence of shared decision-making, collegjallty, and an 
Internal capacity for self-assessment, data-based planning and 
the accomplishment "of objectives. 

b. Evidence of the presence to a high degree of the seven 

characteristics of effective schools. 

«> 

B. Phase II - Val1dat1on/recog^1t1ort criteria would emphasize 

■ ■ \ ' 
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Instructional- outconj*! and Instructional effectiveness based upon the 
Department's definition of an effective schools a school that brings 
children from low-Income families to the. minimum basic skills mastery 
level which now describes minimally successful performance for 
middle-Income children. 

It 1s recommended further that resources be made available to schools to 
help them progress from Phase I to Phase II status. 
Reconmendatlon 5; Review of Instructional Practices 1n Reading 
The evaluation results suggest that the discrepancy 1n reading achievement 
between low-Income and all other children 1s wide and that reading scores for 
low-Income students are significantly lower than scores for all other children. 

It 1s recommended that the nature of reading Instruction 1n the state be 
reviewed and that state leadership be provided to target Improved reading 
Instruction to low-achieving students. 
Recommendation 6; Subgroup Data Analyses 

Statewide mastery testing will make a contribution to the monitoring of 
student progress at the state, district, school and classroom levels. 
However, 1n order to determine whether* or not all students are learning, data 
must be collected and analyzed for subgroups of students. In the School 
Effectiveness Program, manual techniques are used to analyze achievement 
scores for low Income children and for all other children. Data may be 
analyzed for other subgroups such as girls and boys, blacks, hlspanlcs and 
other ethnic groups, and students In special programs such as Chapter I. Each 
school district has Its unique set of subgroups of students for whom subgroup 

analyses may apply. 

It 1s recommended that the Department develop, the technical procedures to 
allow each district and school to analyze mastery test data for particular 
subgroups. The Department would develop the system and provlde-technlcal 
assistance. The district and school would designate the appropriate subgroups. 

68 , 



-60- 



0 

ERJC 



Recommendation 7: Assessment of Organizational Structure 

School effectiveness has no legislative mandate and no clear status within the 

priorities set by Connecticut's Challenge. 

It 1s recommended that the Department examine the policies, allocation of 
resources, management functions, and future directions for school 
effectiveness. 

Recommendation 8: Department Impact 

School effectiveness Ideas are alive and well and the research base continues 
to grow. The Ideas loom larger than any one program and, 1n fact, have -been 
extended and expanded 1n many of the Department's new Initiatives. 

It is recommended that the Department promote collaboration and 
Integration among: 

district-focused Improvement (mastery testing, priority school 
districts); * 

school-focused Improvement ^effective schools); and 

o 

classroom-focused Improvement (curriculum and Instruction, staff 
development). 

Each focus and contribution may be unique but 1t 1s the combined effects 
of district, school and classroom Improvement that will have the greatest 
Impact on Connecticut students. 
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Connecticut School Effectiveness interview 
SAMPLE ITEMS 



• SAFE AN D ORDERLY ENVIRONMENT 

There is an orderly; purposeful atmosphere which is free from the threat of physical harm, 
not oppressive and is conducive to teaching and learning. 



However, the atmosphere is 



is this school a safe and secure place to work? 



The school is not safe 
and secure. Fear and 
concern for physical 
safety are present. 



There is a general 
feeling of insecurity. 
It is not safe to be 
alone in the build- 
ing and numerous in - 
ci dents occur. 



The school is secure 
from outside inter- 
ference. There are 
occasional incidents 
that heighten con- 
cern throughout the 
building. 



There are some 
internal student 
related problems. 
However, adults 
and students gen- 
erally feel secure. 



This is a secure 
building. Students 
and staff do not 
view security as 
an issue. 



XJ 
XJ 
CO 
=3 

ex 



CL EAR SC H OOL HISS10 N 

— . 

There is a clearly-articulated mission for the school through tfhich the staff shares an understanding of and a commitment 
to instructional goals, priorities, assessment procedures and accountability. 



Is there a v/ritten statement of purpose for this sc hool^ that guides the instructional program? 



There is no agreed 
upon, Written stdte- 
iwnt of purpose. 
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A written statement 
exists, 6ut it has 
little influence on 
the instructional 
program. 



A statement of 
purpose has beea 
developed by admin- 
istration and facul- 
ty of this school 
A few general in- 
structional decisions 
are guided by this 
statement* 



A statement 
exists and some 
school decisions 
result from it. 



The statement is 
the driving force 
behind tnost im- 
portant school 
decisions. 
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Describe how time allotments In basic skill areas are determined in this school- 



Individual teachers 
determine their 
own schedules. 



General guidelines are 
handed down by the 
administration. 4 
Teachers develop 
schedules in 
partial compliance. 



The principal develops The principal and 
a general schedule. teachers agree 
Reconwendcd time on allocated 

allotments are generally times'. Schedules 
followed. arc reviewed, 

monitored and/or 
adjusted If 
necessary. 



Allocated time 
in each basic skill 
areas is set with or 
by the principal. 
Teachers and 
principal value 
and monitor these 
time allotments. 



f 



FREOUft lT H0H1T0R1NG OF STUD ENT PROGRESS- 

Feedback on student academic progress is frequently obtained, Hultlple assessment methods such as tether-made tests, 
ctS»c nf fituSK 9 wrk mastery skills checklists, criterion-referenced tests and norm-referenced tests are used. 
iPrSuUs Jf testing are ilsed to 1^ Mi vidua! student perfonuance and also to inwove the instructional program. 



How do you use the information obtained from skill tests, unit tests and/or chapter tests in the basic skills In your 
classroom? 



Information is used 
primarily to give 
students grades. 



Information 1s used 
for grading and making 
groups. There is 
little individual 
feedback beyond 
grades. 



Information Is used 
for grading and to 
plan general class- 
room lessons. The 
Information Is not 
used to modify 
Instruction. 



Information Is 
used to plan * 
lessons for class- 
room groups and to 
give general feedback. 
Instruction is some- 
what modified based on 
resul ts , 



Information is used 
to give specific 
student feedback and 
to diagnose and pre- 
scribe appropriate in- 
struction. 
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Appendix B 

Connecticut School 
Effectiveness Questionnaire 

SAMPLE ITEMS 



5 

in T3 



0J 
< 



e ■£ 

cn w 



1. SAFE Alio ORDERLY ENVIRONMENT 

Teachers, eds'nistrators and parents assume responsibility 

for discipline in this school 5D D U A SA 

2. CLEAR SCHOOL MISSION 

In ynathercatics, written sequential objectives exist uo 
.through all grades.. , SD 0 .11 A SA 

3. INSTRUCTIONAL LEADERSHIP 

The principal leads frequent formal discussion concerning 

instruction and student achievement.... SD D U A SA 

4. HIGH EXPECTATIONS 

Ninety to one hundred percent of the students are expected 

to master all basic skills at each grade level SD D U A- SA 

5. OPPORTUNITY TO LEARN AND STUDENT TIMEON TASK 

Two hours or more are allocated for reading/language arts 

each day throughout this school-. SD D U A SA 

6. FREQUENT MONITORING OF STUDENT PROGRESS 

Teachers and the principal thoroughly review and analyze 

test results'to plan instructional program modifications. . . SD D U A SA. 

7. HOME SCHOOL RELATIONS 

Most parents understand and promote/the schools' instruc- 
tional program SD 0 U A SA 
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Appendix D 
THE CONNECTICUT ARCHIVAL DATA PROFILE. 



Sample 



Principal 
Date: 



Building: 



i and Orderly Environment 
Discipline Policy 


DISTRICT 


, BUILDING | 


1 

COMMENTS 










Infraction Data 














i 1 ' 


Vandalism. Data 









II Clear School Mission 
1 . Statement of Purpose 



2. Written, Sequential 
, Objectives 



3. Mastery Requirements 



III Instructional Leadership 

1. Formal Observation Format 



2. Instructional Planning 
Gui del ines 



3. Staff Development 
Program 



IV High Expectations 

1. Promotion and Retention 
Policy 



2. Retention Data 



3. Grouping Practices 



4. Student Performance 
Reward System 



V Opportunity to Learn and 
Student Time on Task 

1. Allocated Times for 
Instruction 



2. Student Attendance Data 



3. Staff Attendance Data 
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CONNECTICUT SCHOOL EFFECT I VENKSS PROJECT 



SAMPLE 



Qualitative Assessment of School Effectiveness Practices 

Profile 

Component 1 : Assessment of School Need 



(3) 

The achievement profile 
information (SES break) and 
the Information from the 
interview and questionnaire 
instruments were all key 
factors in determining the 
areas needing improvement in 
the school. 



a) Awareness 

The team presented their 
plan to the faculty; nearly 
all of the staff members can 
state the goals of the plan* 



(2) 

One of the sets of infor- 
mation (the achievement 
profile or the data from the 
instruments) provided the 
impetus for diagnosis of 
need. 



(2). f * 

The team made a presentation 

to the faculty, but about 

half of the staff members 

can not state the goals. 



(1) 

Diagnosis of need for 
improvement was based on a 
need perceived by the staff 
which didn't directly relate 
to the characteristics or to 
student achievement* 



(1) 

The plan developed by the 1 
team focused on "an area 
that did not directly 
address any of the seven 
characteristics . 



(1) 

No presentation was made to 
the faculty 9 and most staff 
members can not state the 
goals* 



Developed by: Nancy Sailer, 
Raymond Pechcone 
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Component 2 : Action Plan Development 



(3) 

The planning team developed 
a plan of action for the one 
or more areas judged by the 
staff as the most critical 
needs and specified a time- 
line and steps for action. 



(2) 

The action planning team 
identified priorities for 
the areas .judged by the 
staff as the most critical 
needs but, did not specify 
steps for action* 



Component 3: Awareness of Action Plan 



APPENDIX F 
TABLE 1 

SAFE AND ORDERLY ENVIRONMENT 



Time 1 and 2 School Means, Variances and t Values 3 



School. Time 


N 


Mean 


SO 


t 


A 1 


28 


3.72 


.42 






24 




• 4c 


4 90** 


B 1 


39 


3.44 


.62 






38 




a a 
.of 


06 


C 1 


29 


3.85 


.42 






20 




• JO 




0 1 


21 


3.66 


.61 






21 


3 97 


« DO 


1.52 


E 1 


23 


3.63 


.50 




• 


18 . 


3.86 


45 


1.50 


F 1 


55" 


3.05 


.63 






42 


2 97 


. 30 


.60 


K , 1 


23 


4.08 


.42 




* 2 


22 




• oy 


2 01* 


* L 1 . 


28 


2.99 


.74 






" 22 


3.07 


.69 


.38 


M 1 


60 


3.21 


.62 




w 


53 • 


3.27 


.51 


.54 


S 

N 1 


29 


3.16 


.68 




2 


31 


3.27 


.S3 


.72 



Connecticut School 
Effectiveness Questionnaire 

* P«.10 Number of Items - 10 

** P«.-05 Alpha Internal Consistency - .87 

Test-Retest - .85 
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TABLE 2 
. CLEAR SCHOOL MISSION 
Time 1 and 2 School Means, Variances and t Values 3 



School 


T 4 mc\ 

j ime 


U 

rl 


nca ii 




SO 


t 


A 


i 

2 


28 
24 


3.68 
4.46 


• 


.30 • 
.37 


8.31** 


5 


1 
2 


37 
38 


3 '.58 
3.86 




.41 
.42 


2.83** 


C 


1 
2 


29 
20 


3,98 . 
4.20 


- 


.50 
.41 


1.65 


0 


1 
2 


21 
21 


3.90 
4.24 


* 


*>.53 
.51 


2.13** 


e" 


1 
2 


23 
18 


3.99 
4.19 ■ 




.28 
.30 


2.17** 


F 


1 
2 


55 
42 


3.80 
3.77 




.55 
.49 


.26 


K 


1 
2 


23 
22 


3.56 
3.92 

• 




.39 * 
.57 


2.48** 


L « 


1 
2 


28 
22 


3.41 
3.47 




.67 
.72 


.28 


M 


1 
2 


60 
53 


2.89 
3.28 




.67 
.58 


3.26** 


N ' 


I 
2 


29 
31 


3.16 
3.27 




.67 
.53 


.72 



Connecticut School 
Effectiveness Questionnaire 

* D-e io Number of Items - 14 

** d--'05 > Alpha Internal Consistency - .90 

Test-Retest - .90 

F-2 • 
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"TABLE 3 
INSTRUCTIONAL LEADERSHIP 
Time 1 and 2 School Heans, Variances and t Values 3 



School 



Time 

1 
2 



N 

28 
24 



Mean 

3.31 
3.65 



SO 

.64 

.51 



t 

2.09** 



37 
39 



3.39 
3.60 



.49 
.64 



1.61 



T 
2 



29 
20 



2.90 
3.07 



.58 
.55 



1.05 



1 
2 



21 
21 



3.26 
3.59 



.65 
.79 



1.49 *'- 



1 
2 



23 
18 



3.99 
4.23 



.29 
.28 



2.65** 



1 
2 



55 
42 



3.74 
3.68 



.56 
.40 



.58 



1 
2 



23 
22 



3.72 
4.15 



.44 
.47 



3.19** 



28 
22 



3.09 
3.10 



.54 

.67 



.10 



60 
53 



2.85 
2.91 



.52 
.50 



.53 



27 
30 



3.29 
3.20 



.47 
.53 



.67 



* p^.10 
** p«.Q5 



Connecticut School 
Effectiveness Questionnaire 

Number of Items - 25 

Alpha Internal Consistency - .93 

Test-Retest - .83 



ERIC 
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TABLE * 
HIGH EXPECTATIONS 
Time 1 and 2 School Means, Variances and t Values 2 



School 


Time 


N 




Mean 


SO 


t 


A 


i 

2 


28 
24 




3.35 
3.70 


.44 
.42 


2.93** 


B 


1 
2 


37 
39 




2.75 
2.95 


.46 

.52 


1.77 


C 


1 

2 


29 
20 


© 


3.33 
3.26 


.37 
.36 


.72 


D 


1 

2 


21 
21 




3.12 
3.38 


.46 

.57 


1.61 


E 


1 
2 


23 
16 




3.15 
3.24 ' 


.39 
.34 


.80 


F 


1 
2 


55 
42 




3.09 
2.97 


.49 
.44 


1.31 


K 


1 
2 


23 
22 




3.23 
3.30 


.25 
.27' 


.90 


I 


1 
2 


28 
22 




2.81 
2.83 


.49 

.51 


.15 


M 


1 
2 


60 . 
53 




3.06 
3.04 


.38 
.37 


.24 


N 


1 
2 


29 
31 




3.09 
3.24 


.38 ' 
.31 


1.69 



* p<.10 
** p<.05 



Connecticut School 
Effectiveness Questionnaire 

Number of Items - 12 

Alpha Internal- Consistency - .55 

Test-Retest - .69 
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TABLE 5 

OPPORTUNITY TO LEARN AND TIKE ON TASK 



Time 1 and 2 School Means, Variances and t Values 3 



School 


Time 


N 


Mean 


SO 


t 


A 


i 


28 


4.05 


.36 






2 


24 


3.99 


.40 


.50 , 


B 


1 


36 


3.38 


- 

.37 






2 


38 


3.52 


.38 


1.49 


C 


1 


29 


3.65 


.23 






2 


20 

- 


3.83 


.31 


2.15** 


D 


1 


21 


3.65 


.35 






2 


21 


3.95 - 


.44 


2.46** 


E 


1 


23 


3.65 


.34 






2 


18 


3.7S» 

• 


.46 


1.04 


F 


1 


55 


3.38 


<= 

.50 






2 


42 


3.37 


.38 




K 


1 


23 


3.67 


.31 






2 


22 


3.83 


.23 


1.95* 


L 


1 


28 


3.29 ' 


.43 






2 


22 " 


3.29 


.54 


.04 


K 


1 


60 


3.30 


.41 


A 




2 


53 


3.31 


.42 


.08 


N 


• 1 


27 


3.33 


.40 






2 


31 


3.48 


.30 ■ 


1.64 



Connecticut School 
Effectiveness Questionnaire 



* P«r-10 Number of Items - 12 

** p<..05 * Alpha Internal Consistency - .66 

Test-Retest - »74 



F-5. 

85 



\ 



TABLE 6 
MONITORING STUDENT PROGRESS 
Time 1 and 2 School Means, Variances and t Values 3 



School 


Cycle 


N 


Mean 


SO 


t 


A 


1 


28 


* 3.79 


.33 


4.42** 




2 


24 


4.21 


.35 


B 


1 


38 


3.57 


.44 


3.14* 


2 


37 


3.91 


.48 


c 


1 


29 


3.84 


. -39 


.96 




2 


20 


3.95 


.39 


0 


1 


21 


3.91 


.45 


1.63 




2 


21 


4.16 


.53 


E 


1 


23 


3.91 


.25 


2.70** 


2 


'• 18 


4.16 • 


.35 


F 


1 


55 


3.73 


.54 


2.04** 

> 




2 


42 


3.53 


.38 


K 


1 


23 


3.75 


.37 


1.65 




2 


22 


3.95 


.44 


I 


1 


28 


3.42 


.56 


1.13 




2 


22 


3.59 


.47 


M 


1 


60 


3.47 


.47 


.87 




2 


S3 


3.55 


.43 


N 


1 


29 


3.57 


.38 


.89 




2 


31 


3.66 


.40 



* p^.io 

** p«s .05 



Connecticut School 
Effectiveness Questionnaire 

Number of Items - 11 

Alpha Internal Consistency - .77 

Test-Retest - .67 



ERIC 



F-6 



86 







TA8LE 7 




■ 






HOME/SCHOOL RELATIONS 








Time 1 and 2 


School Means, Variances 


and t Values 3 




C r* hnnl 

ocnoo i 


Time N 


Mean 

\ * 


SO 


t 


A 


1 28 


3.48 


.47 






2 24 


3.97 

• 


.37 


4.11** 


o 
D 


1 37 


2.61 


.44 






2 37 


2.85 


.51 


2.15* 


c 


1 29 


3.36 


.37 






2 20 


3.46 


.28 


1.07 


U 


1 21 


3.31 


. .56 






2 21 


3.65 


.54 


2.03* 


r 
t 


1 23 


3.31 


.42 






2- 18 


3.49 


.44 


1.38 


r 
r 


1 55 


2.99 


.51 






1 2 42 


2.86 


.44 


1.27 


V 

K 


1 23 


3.61 


.40 1 






,. 2 22 


3.90 


.44 J 


2.28** 


L 


• 

1 28 


2.50 


.54 






. 2 22 


2.84 


.61 


2.06** 


M 


1 60 




.32 






"2 53 


-3.00 


.33 


2.61** 


H 


1 28 


2.93 


.49 






2 31 


2.95 


.41 


.17 











Connecticut School 
Effectiveness Questionnaire 



* P <-J2 Number of Items - 15 

P<- 05 " Alpha Internal Consistency - .88 

Test-Retest - .82 



' ERJC .. 



Appendix G 

A Description of a Hethod for Aggregating Achievement Data 



Due to the variety of standardized tests used across schools and .the 
variety of grade levels tested within schools, a method for aggregating all 
the test data across schools was needed. For purposes of evaluation the 
following Index of school effectiveness was created: 

Step 1: The frequency and proportion of all low Income student scoring 
below the thirtieth percentile (proficiency standard) were 
tallied within each school. The frequency and proportion of 
other students was also tallied for purposes of comparison. 

Step 2: The frequency and proportion of all low Income students scoring 
below the fiftieth' percentile (grade level) was recorded 1n each 
school. These data were also collected for other students. 

Step 3: The frequency and proportion of all' low Income students scoring 
above the fiftieth percentile (above grade level) was recorded 1 
each scjioql. These data were also collected for other students. 

Step 4: The data at each of the three cutoff, (30th percentile, 50th 

percentile, and 50* percentile) were then aggregated across all 
schools. Weighted means were' calculated to more accurately 
reflect school based Impact by taking Into account (weighting) 
the schools on the basis of Its total number of students 1n 
Income classification. > J~\ 

Using the above method accomplishes two Important functions (1) the data 
1s more manageable thus trends 1n the, data can be more efficiently displayed 
and (2) 1t provides a method for combining and displaying data within and 
across schools. 
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Table 10 , 

Aggregate Reading Achievement: The Relationship of Time on the 
Performance of Low Income and Other Students 

Reading * 

Percent Below 3QXtle Critical 
Group Comparisons Total N Cycle 1 Cycle 2 Cycle 3 SE . Value 



low Income 



a. Cycle 1/Cycle 2 


771 


.41 


.40 


.025 


.40 


b. Cycle 2/Cycle 3 


830 




.40 


33 .024 


2.87** 


c* Cycle 1/Cycle 3 


771 

<> 


.41 




33 .025 


3.21** 


Other Income 












a. Cycle 1/Cycle 2 


581 


.16 


.14 


.020 


.98 


b. Cycle 2/Cycle 3 


650 




.14 


16 .020 


-.98 


c. Cycle 1/Cycle 3 


581 


.16 




16 .021 


.066 


- Assessing the Discrepancy Between Low Income and Other Students* 




Low Income/Other 


N 


Low Income' 


Other 


SE 


CV 


a. Cycle 1 


771 


.41 


< .16 


.025 


9.8£** 


b. Cycle 2 


830 


.40 


.14 


.024 


10.99** 


c. Cycle 3 


748 


.33 


.16 


.024 


7.11** 



' Group Comparisons - Test administration over three years: Cycle 1 (SY 

1982), Cycle 2 (SY -1983) and Cycle 3 <SY 1984). 
Total N - Total number of low and other Income children 

Income (Low and Other) - Percent of low Income and other students scoring 

° - below thfe prof iciency standard. 

SE - Standard error 

♦These analyses assesses whether the gap between the performance of low Income 
students 3s significantly different from all other students below the 
proficiency standard. No significant difference 1s a desirable finding. 
"Critical value - p<.01 



H-l 
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Table 11 



Aggregate Mathematics Achievement: The Relationship of Time on the 
Performance of Low Income and' Other Students 



Group Comparisons 



Total N 



Mathematics 

Percent Below 30yile 
Cycle 1 Cycle 2 Cycle 3 



SE 



Critical 
Value 



Low Income 

a. Cycle 1/Cycle 2 736 

b. Cycle 2/Cycle 3 865 

c. Cycle 1/Cycle 3 736 

Other Income 

a. Cycle 1/Cycle 2 571 

b. Cycle 2/Cycle 3 650 

c. Cycle 1/Cycle 3 571 



.36 
.36 

.17 
.17 



.22 
.22 



.16 
.16 



.24 

.23 



.14 
.14 



.23 

.023 

.024 



.021 
.019 
.020 



6.19* 

-.97 

5.13** 



.46 
1.01 
1.45 



Assessing *the Discrepancy Between Low Income and Other Students 



Low Lncome/Other 



Low Income Other 



a. Cycle 1 

b. Cycle 2 

c. Cycle 3 



736 
864 
792 



.36 
.22 
.24 



.17 
.16 
.14 



.025 7.62** 
.020 2.92** 
.021 4.81** 



Key 

Group Comparl 

Total K 
Income (Low a 

SE 

•These analys 
Students 1s s 
proficiency s 
••Critical va 



sons - Test administration over three years: Cycle 1 (SY 
1982), Cycle 2 (SY 1983) and Cycle 3 (SY 1984). 

- Total number of low and other Income children 

nd Other) - Percent of low Income and other .stents scoring 
below the proficiency standard. 

- Standard error • 

es assesses whether the gap between the performance of low Income 
1gn1f1cantly different from all other students below the 
tandard. No slgnlf leant ^difference 1s a desirable finding, 
lue - p^.01 
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Table 12 

Evaluating Implementation: The Relationship of Time and the Reading Performance 

of Low Income and Other Students 
Reading \ 



Group Comparisons 



Percent Below 30X1lg 
Total H Cycle 1 Cycle. 2 Cvcle 3 SE 



Critical 
Value ' 



Moderate Incrementation 
low Income 

a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 

Other Income 

a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 

High Implementation 
Low Income \ 

a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 

Other Income 

a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 



227 
262 
227 



54 

50 
54 



37 
39 
37 



24 
27 
24 



.45 
•*5 

.17 
.17 



.30 
.30 

.13 
.13 



.44 

.44 



.1.2 
.12 



.38 
.38 



.11 
.11 



.35 
.35 



.15 
.15 



.29 



.14 
.14 



.030 
.029 
.030 



.026 
.024 
.028 



.063 
.066 
.061 



.031 
.034 
.037 



.33 
3.11** 
3.31** 



1.92 
■1 .01 
.71 



■1.23 
1.34 
.18 



.62 
-.88 
.27 



Key. 

Group Comparisons 
Tot2>l N 

Income (Low and Other) - 
SE 

"Critical value - p<.01 



Test administration over three years: Cycle 1 (SY 
1982), Cycle 2 (SY 1983) and Cycle 3 (SY 1984). 
Total number of low and other Income children 
Percent of low Income and other students scoring 
below : the proficiency standard., 
Standard error 
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Table 13 

Evaluating Implementation:, The Relationship of Time and the Mathematlc 
Performance of Low Income and Other Students 

Mathematics 







Percent 


Below 


StfrUe 




Critical 


Group Comparisons 


Total N 


Cycle 1 


Cycle 2 


Cycle 3 




Value 














Moderate Implementation 














Low Income 














a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


191 
130 
191 


.38 
.38 


.25 
.25 


.22 
.22 


.029 
.026 
.028 


4.47** 

1.16 

5.69** 


Other Income 














a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


64 
64 
64 


.22 
.22 


.18 
.18 


.15 
.15 


.032 
.027 
.030 


1.30 
1.08 
2.34** 


H^ah Tmnl#mentat 1 on 














Low Income 














a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


37 
33 
37 


.33 
.33 


.24 
.24 


:i7 

.17 


.057 
.052 
.057 


1.60 
1.35 
2.78** 


Other Income 




> 










a. ^ Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


36 
26 
36 


.14 
.14 


.11 
.11 


.08 
.08 


.029 
.024 
.025 


1.04 
1.25 
2.38** 



Key 

Group Comparisons - Test administration over three years: Cycle 1 (SY 

1982). Cycle 2 (SY 1983) and Cycle 3 (SY 1984). 

Total H - Total number of low and ether Income children 

Income (Low and Other) - Percent of low Income and other students scoring 

below the proficiency standard. 

SE - Standard error 

**Cr1t1ca1 value - p^.01 



9 

ERIC 
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Table 14 

Evaluating Leadership: The Relationship of Time and the Reading Performance 

of Low Income and c other students 

Reading 



Group Comparisons 


Total N 


Percent Below 30*1 le 
CVCle 1 Cvcle 7 Pvcle 3 


SE 


Critical 
Value 
















Moderate Leadership 














Low Income 














a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


92 
83 
92 


.25 ' 
.25 


.25 
.25 


.23 
.23 


.033 
.033 
.032 


0 

.60 
.62 


Other Income 






• 








a. Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


57 
37 
57 


.20 
.20 


.15 
.15 


.18 
,18 


.033 
.032 
.033 


1.51 
-.94 
.60 


Average Leadership 














Low Income 














a. Cycle 1/Cycle 2 

b. ' Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


224 
249 
224 


.47 
.47 


.45 
.45 


.40 

.37 


.031 
.031 
.032 


.65 
2.58** 
.3.12** 


Other Income 














a. . Cycle 1/Cycle 2 

b. Cycle 2/Cycle 3 

c. Cycle 1/Cycle 3 


36 
54 
36 


.09 
.09 


,13 
.13 


.13 
.13 


.022 
.025 
.023 


-1 .83 

-0.002 

-1.75 



£21 

Group Comparisons - Test administration over thrie years: Cycle 1 (SY 

1982), Cycle 2 (SY 1983) and Cycle 3 (SY 1984). 

Total N - Total number of low and other Income children 

Income (Low and Other) - Percent of low Income and other students scoring 

below the proficiency standard. 

SE - Standard error 

"Critical value - p-c^.01 
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ERIC 
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Table 15 



Evaluating Leadership: The Relationship of Time, and the Hathematlc 
Performance of Low Income and Other Students 

Mathematics 



Group Comparisons 



Percent Below 30%1le 
T otal N Cvcle 1 Cycle 2 Cvcle 3 



Critical 
SE Value 



Moderate Leadership 




■ 








Low Income 








i 




a. Cycle 1/tycle 2 


79 


.28 


.23 




.035 


b. Cycle 2/Cycle 3 


74 




.23 


.27 


.034 


c. Cycle 1/Cycle 3 


79 


.28 




.27. 


-,-03.6 


utner income 












a Purl** 9 


w / 


n? 


n? 


• 




b. Cycle 2/Cycle 3 


52 




.02 


.16 


..032 


c. Cycle 1/Cycle 3 


67 


.02 




.16 


.030 


High Leadership 












Low Inc'ome 






» 






a- Cycle 1/Cycle 2 


186 


.39 


.22 




.029 


b. Cycle 2/Cycle 3 


116 




.22 


.22 . 


.027 


c. Cycle 1/Cycle 3 


186 


.39 " 




.22 


.030 


Other Income 












a. Cycle 1/Cycle 2 


30 


.090 


.12 




.023 


b. Cycle 2/Cycle 3 


52 




.12 


.12 


.024 


c. Cycle 1/Cycle 3 


30 


.090 




.02 


.024 



1.42 
-1.19 
.28 



0 

1.28 
1.36 



5.86* 
0- 

.5.60* 



-1.33 
0 

-1.26 




Group Comparisons - Test' administration over three years: Cycle 1 (SY 

> 1982)', Cycle 2 (SY 1983) and Cycle. 3 (SY 1984). 
Total N - Total number of low and other Income children 

Income (Low and Other) - Percent of low Income and other student^ ^coring 

• below the* proficiency standard. 
SE - Standard error 

**Cr1t1cal value - p<.01 
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Table 16 - 

The Significance of the Discrepancy Between Low Income and Other Students 
1n Reading and Mathematics for Two Levels of Implementation 

Reading *^ 



Income 

_N Low Other Gap 



Moderate 
Implementation, 

Cycle 1 yill 
1 262 
i 189 

High 

Implementation 



.45 
.44 
.35 



.17 
.12 
.15 



.28 
.32 
.20 



SE 



CV 



.034 
.029 
.030 



8.26** 
10.86** 
6.67** 



Cycle 1 
2 
3 



37 
39 
29 



.30 
.38 
.29 



.13 
.11 
.14 



.17 
.27 
.15 



.046 
.046 
.052 



3.69** 
5.83** 
2.90** 



Mathematics* 



N 



Moderate 
Implementation 



Income 

Low Other Gap SE 



CV 



Cycle 1 
2 
3 



191 
130 
123 



.38 
.25 
.22 



.22 
.18 
.15 



.16 
.07 
.07 



.034 
.029 
.026 



4.62** 
2.46** 
2.65** 



High 

- Implementation 



Cycle 1 
2 
3 

m 

Group Comparisons 



37 
33 
19 



.33 
.24 
.17 



.14 
.11 
.08 



.19 
.13 
.09 



.045 
.038 
.033 



4.22** 
3.38** 
2.73** 



- Test administration over three years: Cycle 1 (SY 
1982). Cycle 2 (SY 1983) and Cycle 3 (SY 1984). 
Total N - Total number of low and other Income children * 

Income (Low; and Other) - Percent of low Income and other students scoring 

below the proficiency standard. 
Gap - The difference between low and other Income students 

scoring below the proficiency standard. < 
SE - Standard error 

*These analyses assesses whether the gap between the performance of low Income 
students 1s significantly different from all other students below the 
proficiency standard. No significant difference 1s a desirable finding. 
**Cr1tft*V value - p<.01 



ERIC 
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Table 17 

Significance of the Discrepancy Between Low Income and Other Students 
1n Reading and Mathematics for Two levels of Leadership 

Reading* 

Income 

N Low Other Gap - SE CV 

Average 
Leadership 

Cycle 1 92 

2 83 

3 78 

High 

Leadership 

Cycle 1 224 

2 249 

3 169 



.25 


.20 


.05 


.033 


1.51 


.25 


.15 


.10 


.034 


2.94** 


.23 


.18 


.05 


.033 


1 .53 

• 


.47 


.09 


.38 


.031 


12.28** 


.43 


.13 


.32 


.030 


10.63** 


.37 


.13 


.24 


.032 


7.54** 



Mathematics * 
Income 





N 


Low 


Other 


Gap 


SE 


CV 


Average 














Leadership 














Cycle 1 


79 


.28 


.20 


.08 


.034 


2.33** 


2 


74 


.23 


.20 


.03 


.034 


.868** 


3 


91 


.27 


.16 


.11 


.032 


3.49** 



High 

Leadership 

Cycle 1 
2 
3 



186 


.39 


.09 


.30 


.032 


9.50** 




116 


.22 


.12 


.10 


.025 


4.06** 




99 


.22 


.12 


.10 


.028 


3.62** 





Group Comparisons - Test administration over three years: Cycle 1 (SY 

1982), Cycle 2 (SY 1983) and Cycle 3 (SY 1984). 

Total N - Total number of low and other Income children 

Income (Low and Other) - Percent of low tncome and other students scoring 

below the proficiency standard. 

Gap - The difference between low and other Income students 

scoring below the proficiency standard. 

SE , Standard error 

♦These analyses assesses whether the gap^between the performance of low income 
students 1s significantly different from all other students below the 
proficiency standard. No significant difference 1s a desirable finding. 
**Cr1t1cal value - p<^01 
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